Skip to content

Commit

Permalink
Fix: Ensure deterministic file ordering in iter_package_files
Browse files Browse the repository at this point in the history
### Summary
Fixed non-deterministic file ordering in the `iter_package_files` function by adding explicit sorting of package files. This change ensures consistent behavior across builds and prevents issues caused by unordered file iteration.

### Changes Made
- Applied `sorted()` to file listings in both Python versions (<3.11 and >=3.11).
- Guaranteed that files are processed in a consistent, lexicographic order.

### Why This Fix?
- Prevents non-deterministic builds caused by unordered file iteration.
- Improves build reproducibility and enables better caching in CI/CD pipelines.
- Simplifies code by enforcing consistent ordering within the function, reducing the need for manual sorting in callers.

### Testing
- Rebuilt the project multiple times and verified that output files are now consistent across builds.
- Compared generated artifacts (e.g., `.json` files) to confirm deterministic ordering.

### Impact
- No functional changes to how files are processed, only improves determinism.
- Safer and more reliable builds, especially in CI environments.

Closes linkedin#265
  • Loading branch information
lilaliu-stripe authored Jan 15, 2025
1 parent a353d10 commit 15d6556
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/shiv/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@
# N.B.: `importlib.resources.{contents,is_resource,path}` are deprecated in 3.11 and gone in 3.13.
if sys.version_info < (3, 11):
def iter_package_files(package: Union[str, ModuleType]) -> Iterator[Tuple[Path, str]]:
for bootstrap_file in importlib_resources.contents(bootstrap):
for bootstrap_file in sorted(importlib_resources.contents(package)):
if importlib_resources.is_resource(bootstrap, bootstrap_file):
with importlib_resources.path(bootstrap, bootstrap_file) as path:
yield (path, bootstrap_file)
else:
def iter_package_files(package: Union[str, ModuleType]) -> Iterator[Tuple[Path, str]]:
for resource in importlib_resources.files(package).iterdir():
for resource in sorted(importlib_resources.files(package).iterdir(), key=lambda r: r.name):
if resource.is_file():
with importlib_resources.as_file(resource) as path:
yield (path, resource.name)
Expand Down

0 comments on commit 15d6556

Please sign in to comment.