Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

actions: add CopyObject action #47

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Sporif
Copy link

@Sporif Sporif commented Jan 26, 2022

No description provided.

@codecov
Copy link

codecov bot commented Jan 26, 2022

Codecov Report

Merging #47 (1b553b6) into main (448047a) will decrease coverage by 0.41%.
The diff coverage is 80.76%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #47      +/-   ##
==========================================
- Coverage   91.53%   91.12%   -0.42%     
==========================================
  Files          26       27       +1     
  Lines        1300     1352      +52     
==========================================
+ Hits         1190     1232      +42     
- Misses        110      120      +10     
Impacted Files Coverage Δ
src/actions/mod.rs 100.00% <ø> (ø)
src/bucket.rs 92.96% <0.00%> (-3.00%) ⬇️
src/actions/copy_object.rs 87.50% <87.50%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 448047a...1b553b6. Read the comment docs.

Copy link
Collaborator

@guerinoni guerinoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your job :)

Copy link
Owner

@paolobarbolini paolobarbolini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I left a change for the implementation and a nitpick for the docs.

self.headers.iter(),
),
None => {
url.query_pairs_mut().append_pair(
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should instead call crate::signing::util::add_query_params(url, self.query.iter()) so that any custom query parameters are also included.

Example from another action:

None => crate::signing::util::add_query_params(url, self.query.iter()),

Comment on lines 11 to 22
/// Create a copy of an object that is already stored in S3, using a `PUT` request.
///
/// Find out more about CopyObject from the [AWS API Reference][api]
///
/// [api]: https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html
Copy link
Owner

@paolobarbolini paolobarbolini Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some actions also mention a few important pieces of information I found from the AWS docs, for this one I think it would be worth to briefly mention some of the footguns with this method:

  • only objects up to 5 GB can be copied using this method
  • even if the server returns a 200 response the copy might have failed

@paolobarbolini
Copy link
Owner

The time crate bumped MSRV again @guerinoni 🙄

@guerinoni
Copy link
Collaborator

@Sporif also rebase on main branch, I fixed the MSRV :)

query.insert(
"x-amz-copy-source",
format!("{}/{}", bucket.name(), src_object),
);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use SortingIterator while signing to save a Map allocation, like here?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By prefixing the current bucket name, this action does not allow copying from a different bucket.

Copy link
Owner

@paolobarbolini paolobarbolini Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use SortingIterator while signing to save a Map allocation, like here?

Sure, sounds like a good use for it.

By prefixing the current bucket name, this action does not allow copying from a different bucket.

We could take both a src_bucket and a dst_bucket

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taking a Bucket is unnecessarily involved, as you need to pass connection details for bucket construction which will go completely unused. How about taking the raw source string, and having two separate functions on Bucket, e.g.:

  • Bucket::copy_object would prepend the current bucket name,
  • Bucket::copy_object_from_bucket would pass the source string as-is, and require the caller to prepend a (potentially different) bucket name

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good

@Sporif
Copy link
Author

Sporif commented Jan 27, 2022

Implemented the changes as best I could, but there's one big problem I hadn't noticed before: the copy doesn't actually work (at least on minio) as the copied object is empty. No idea why.

@Sporif Sporif changed the title actions: add CopyObjects action actions: add CopyObject action Jan 27, 2022
@Sporif
Copy link
Author

Sporif commented Feb 1, 2022

Ok so I'm really dumb, none of this is needed, x-amz-copy-source is just a header to add to a put_object request. Not sure it's worth adding a test for that and/or some sort of helper function, or should I just close this PR.

@paolobarbolini
Copy link
Owner

Ok so I'm really dumb, none of this is needed, x-amz-copy-source is just a header to add to a put_object request. Not sure it's worth adding a test for that and/or some sort of helper function, or should I just close this PR.

I think it'd be still worth it to have it as a separate method. I'll try looking into why the test fails

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants