From c37b9f5939391f1846d4302f4929756ed53891e3 Mon Sep 17 00:00:00 2001
From: Daniel Stenberg <daniel@haxx.se>
Date: Thu, 19 Dec 2024 17:28:25 +0100
Subject: [PATCH] skip: Skip download if already done
Fixes #505
---
SUMMARY.md | 1 +
usingcurl/transfers/README.md | 1 +
usingcurl/transfers/skip.md | 24 ++++++++++++++++++++++++
3 files changed, 26 insertions(+)
create mode 100644 usingcurl/transfers/skip.md
diff --git a/SUMMARY.md b/SUMMARY.md
index 58702df3f..4bd03570f 100644
--- a/SUMMARY.md
+++ b/SUMMARY.md
@@ -104,6 +104,7 @@
* [Rate limiting](usingcurl/transfers/rate-limiting.md)
* [Request rate limiting](usingcurl/transfers/request-rate.md)
* [Compression](usingcurl/transfers/compression.md)
+ * [Skip download if already done](usingcurl/transfers/skip.md)
* [Connections](usingcurl/connections/README.md)
* [VLAN](usingcurl/connections/vlan.md)
* [Name resolve tricks](usingcurl/connections/name.md)
diff --git a/usingcurl/transfers/README.md b/usingcurl/transfers/README.md
index 082d3ed33..6d390819e 100644
--- a/usingcurl/transfers/README.md
+++ b/usingcurl/transfers/README.md
@@ -8,3 +8,4 @@ multiple transfers.
* [Rate limiting](rate-limiting.md)
* [Request rate limiting](request-rate.md)
* [Compression](compression.md)
+ * [Skip download if already done](skip.md)
diff --git a/usingcurl/transfers/skip.md b/usingcurl/transfers/skip.md
new file mode 100644
index 000000000..f2ba420b6
--- /dev/null
+++ b/usingcurl/transfers/skip.md
@@ -0,0 +1,24 @@
+# Skip download if already done
+
+Sometimes when writing a script or similar to do internet transfers every now
+and then you end up in a situation when you rather have curl skip doing the
+transfer if indeed the target file is already present locally. Maybe after the
+previous invoke, maybe for another reason.
+
+This kind of extra logic can certainly be added in common shell script logic
+before curl is invoked in many cases but not all. For example things get
+complicated when you ask curl to do [globbed download
+ranges](../../cmdline/urls/globbing.md).
+
+For example, download one thousand images from a site but skip the ones that
+we already have downloaded previously:
+
+ curl --skip-existing -O https://example.com/image[0-999].jpg
+
+It should be noted that this only checks the *presence* of the local file, it
+makes not attempts to verify that it actually has the correct content and it
+has no way to do any such checks.
+
+curl offers other options to do [conditional
+transfers](../../http/modify/conditionals.md) based on modified date or
+contents.