Handle storage failures more gracefully

[mprasil]

I had my rustic copy operation fail multiple times with some variation of (I have truncated the long IDs):

[WARN] operation=Writer::close path=data/a7/a79b7... -> retry after 1.347199082s: error=Unexpected (temporary) at Writer::close, context: { url: https://s3.eu-west-1.amazonaws.com/bucket/data/a7/a79b7... , called: http_util::Client::send, service: s3, path: data/a7/a79b7... } => send http request, source: error sending request for url (https://s3.eu-west-1.amazonaws.com/bucket/data/a7/a79b7... ): connection closed before message completed

Then rustic exits with non-zero exit code. Retrying the copy command works as expected and the copy eventually succeeds, but perhaps this could be handled more gracefully with multiple retries?

This is with rustic 0.7.0 using S3 backend.

[aawsome]

by default, rustic retries 5 times until it gives up. You can customize this using

[repository.options]
retry = "25" # or any other number

UPDATE: This is the retry per individual read or write access to the backend. Not the total number of retries.

[mprasil]

Ah right, I haven't noticed the option in the example configuration. Thank you!