Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Don't overwrite already downloaded files but delete/flag unfinished files #8

Open
Noctunus opened this issue Mar 18, 2019 · 2 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@Noctunus
Copy link

When for an arbitrary reason the download of a series has failed and is re-started from scratch then already downloaded files are completely downloaded again and overwritten.

It would be great if a check would be implemented if a file is already present and then skip the download. This could also be activated / skipped by a flag if it's more convenient for other users to keep the default behavior.

Another thing is - in case already present files are not overwritten then partly downloaded files would also be kept but in my opinion they should always be deleted in an error case or at least marked as unfinished. To be 100% crash-safe on this end one could think of a temporary filename for the download process (just like the name + ".tmp") and rename it to the desired filename after success.

@Noctunus Noctunus changed the title Feature Request: Don't overwrite already downloaded files but delete unfinished files Feature Request: Don't overwrite already downloaded files but delete/flag unfinished files Mar 18, 2019
@simplymemes simplymemes added enhancement New feature or request help wanted Extra attention is needed labels Apr 15, 2019
@Platypuschan
Copy link

Mhm i´m just a scriptkiddie, but what´s about a more elaborated thingy like

if $already-existing-file-size != $size-of-file-to-download {
    rm $already-existing-file && download $new-file

@Noctunus
Copy link
Author

Noctunus commented Nov 5, 2019

For this the target file size has to be known up-front and at least the HTTP standard allows chunked transfer of files where you only know that the file is completed if it's returned as completed in the last chunk. Therefore a fail-proof way to receive files would be to save it to a temporary filename and only if it's finished successful rename it to the target filename. This would solve several things at once:

  1. You can be sure that the target filename is a complete file instead of only a part in an error case.
  2. If the target filename is already present you can just skip it without looking at sizes.
  3. Resuming a download would be possible as you know that the temp filename only consists of a part - to keep it simple you could just skip this and re-download the whole file with the same temp filename just overwriting the old data.

Also the implementation should be pretty simple (assuming the target filename is $name) :

  1. If $name already exists just skip the current file
  2. For the downloading just add a ".tmp" as suffix
  3. On successful download rename $name.tmp to $name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants