pack xxxxxx not referenced in index

[neugartf]

Hey!

I used a hot/cold repo to backup files of mine to Scaleway.
After finishing the backup, I ran a rustic check which resulted in warnings spilling out

[WARN] pack b48c7881 not referenced in index. Can be a parallel backup job. To repair: 'rustic repair index'.
[WARN] pack b644cc40 not referenced in index. Can be a parallel backup job. To repair: 'rustic repair index'.
[WARN] pack bf6a13ff not referenced in index. Can be a parallel backup job. To repair: 'rustic repair index'.
[WARN] pack d5e1440c not referenced in index. Can be a parallel backup job. To repair: 'rustic repair index'.
[WARN] pack f2e9c507 not referenced in index. Can be a parallel backup job. To repair: 'rustic repair index'.
[WARN] pack f31d285e not referenced in index. Can be a parallel backup job. To repair: 'rustic repair index'.

I then tried to run rustic repair index but also this ran into 403 for those hashes.

I'm obviously worried that the backup is somewhat incomplete. How can I fix those warnings?

Very much thanks! :)

---

Rustic version: 0.9.5

[repository]
repository = "opendal:s3"
repo-hot = "opendal:s3"
password = ""

[repository.options-cold]
endpoint = "https:/coldbucket.s3.fr-par.scw.cloud"
bucket = "coldbucket"
region = "fr-par"
root = '/'
default_storage_class = "GLACIER"
access_key_id = ""
secret_access_key = ""

[repository.options-hot]
endpoint = "https://hotbucket.s3.fr-par.scw.cloud"
bucket = "hotbucket"
region = "fr-par"
root = '/'
default_storage_class = "STANDARD"
access_key_id = ""
secret_access_key = ""

[[backup.snapshots]]
sources = ["/mnt/external-drive"]

rustic list packs  | rg f31d285e
PackId(f31d285e0160e0249a947e7f8d350b11fdbb16a77bd6addaa482e2e562898974)

rustic list indexpacks  | rg f31d285e
EMPTY

[aawsome]

Hi @neugartf

Can you please post the full output of rustic check? These warnings alone are - just warnings, they only mean you have additional pack files which may be result of an aborted or parallel backup. If there are no other warnings or errors, there is nothing to worry.

About rustic repair index: This needs to read the pack files, which is obviously forbidden in your case. Could it be that your cold storage forbids read access? In this case you have to specify a warm-up-command in order to be able to warm-up and then read the needed data. However, if these warnings are the only findings of check, you may also simply run a rustic prune which should also just delete these pack files without the need to access them.

[neugartf]

Hey @aawsome!

Thanks for the support!

Indeed a rustic prune did remove most of the warnings, except this one (this may be even new):

[WARN] hot pack 87869e0a is a data pack. This should not happen.

---

[INFO] using config /home/pi/.config/rustic/rustic.toml
[INFO] repository opendal:coldbucket#opendal:hotbucket: password is correct.
[INFO] using cache at /home/pi/.cache/rustic/d21ae4798b447ae16268cf87eebfc0d53999c9cab4931a02715f7a46fc4d0588
[00:00:00] getting snapshots...           ████████████████████████████████████████          1/1                                 [00:00:00] checking Snapshot in cache...  ████████████████████████████████████████      522 B/522 B      523.16 KiB/s (ETA -)   [00:00:00] checking Index in cache...     ████████████████████████████████████████  16.35 MiB/16.35 MiB  249.53 MiB/s (ETA -)   [00:00:00] checking Snapshot in hot repo...                                                                                    
[00:00:00] checking Index in hot repo...                                                                                        
[00:00:01] reading index...               ████████████████████████████████████████         11/11                                [00:00:00] listing packs in hot repo...   ⠁                                                                                     [WARN] hot pack 87869e0a is a data pack. This should not happen.
[00:00:00] listing packs in hot repo...                                                                                         
[00:00:02] listing packs...                                                                                                     
[00:00:00] cleaning up packs from cache...                                                                                      
[00:00:09] checking packs in cache...     ████████████████████████████████████████  45.72 MiB/45.72 MiB  4.75 MiB/s   (ETA -)   [00:00:08] checking trees...              ████████████████████████████████████████          1/1