Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to verify SHA256 hash during S3 downloads #212

Open
an-presley opened this issue Jan 16, 2025 · 1 comment
Open

Add option to verify SHA256 hash during S3 downloads #212

an-presley opened this issue Jan 16, 2025 · 1 comment

Comments

@an-presley
Copy link

Currently, when writing data to S3, tensorstore includes the data's SHA256 hash in the x-amz-content-sha256 header. From the docs, it looks like S3 returns this hash during downloads, but tensorstore doesn't currently verify it against the downloaded data. It would be helpful if there was an option to verify it, to build confidence there is no corruption when reading tensors from S3.

@laramiel
Copy link
Collaborator

Something to be aware of for #114

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants