Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issues of Fusio OpenDAL Integration #91

Open
Xuanwo opened this issue Nov 2, 2024 · 0 comments
Open

Tracking issues of Fusio OpenDAL Integration #91

Xuanwo opened this issue Nov 2, 2024 · 0 comments

Comments

@Xuanwo
Copy link
Contributor

Xuanwo commented Nov 2, 2024

This issue is used to track future work on Fusio OpenDAL Integration.

Tasks

Integrate with fusio's own HttpClient

Fusio has its own HttpClient abstraction that can integrate with different async runtimes. We can integrate OpenDAL with this HTTP client to avoid relying on reqwest instead.

Visit opendal's HttpFetch for more information.

Integrate better with opendal's buffer/concurreny

OpenDAL offers excellent support for buffering and concurrency, making it very easy to upload a large file in 16 parts.

let w = op.writer_with(path).buffer(8 * 1024 * 1024).concurreny(16).await?;
w.write(data).await?; // repeat writing data...
w.close().await?;

fusio::Fs doesn't offer such features for opendal to adjust those settings; perhaps we can configure them using OpendalFs instead.

create_dir_all doesn't implement correctly

fusio::Fs::create_dir_all doesn't take &self which makes it's impossible for fusio_opendal to implement it correctly.

Docs and examples

fusio_opendal can serve as an excellent fallback for any services we do not support for now. Perhaps we can include this in our documentation and add corresponding examples.

Unit tests and integration tests

We can implement unit tests using opendal's memory service and possibly add integration tests for other services like S3, GCS, or AzBlob, depending on our users' needs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant