Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature add bun orm lib to simplify data access #480

Closed
wants to merge 6 commits into from
Closed

Conversation

evrins
Copy link
Collaborator

@evrins evrins commented May 21, 2024

since we are supporting multi type database it will make much more easier to access data with orm.
with orm we won't have to write the same sql multi times
also the bun lib api looks good to me

the bigest changes is the underlaying sqlite lib bun sqlite lib
it will use cgo free modernc.org/sqlite on supported platforms
and mattn/go-sqlite3 if Cgo is enabled

I've tested compatibility between old and new verison of exatorrent, it works fine for both version

@evrins evrins requested review from nlevee and varbhat and removed request for nlevee May 21, 2024 08:32
@varbhat
Copy link
Owner

varbhat commented May 21, 2024

@evrins ,

I am not comfortable with using ORM for DBs in exatorrent. It would bloat the binary size and I prefer SQL and not the ORM abstraction. (I hold the opinion that ORMs are bad. Lot of bugs and complexity arises from ORMs and it gets hard to debug. That's why I've avoided it at first place when I was developing. And I still hold this opinion)

I just tested your branch right now. Changes you've made has bloated the binary by 2MB on Linux. Not worth it if the changes don't add any features.

But, sqliteshim which automatically imports SQL libs seem to be interesting. And to be honest, I did reject the modernc sqlite driver during the development of sqlite.

For the Sqlite, i think that it would be better to stick with https://github.com/go-llsqlite/crawshaw (see crawshaw's article on this) or mattn-sqlite.

For the CGo less port, I would prefer some Golang Keyval Datastore libs (no boltdb please!)

Also, Note that I am planning of huge overhaul to the Database code(I plan to add storage drivers too which abstracts torrent storage). I just need to get time to work on this.

@varbhat varbhat closed this May 21, 2024
@varbhat
Copy link
Owner

varbhat commented May 21, 2024

Yes, I agree that tests need to be written for DB.

sqlite doesn't need any external tools for testing.

Let us use https://github.com/fergusstrange/embedded-postgres for psql tests.

@varbhat
Copy link
Owner

varbhat commented May 21, 2024

@evrins , Sorry for rejecting this PR. I appreciate your work. Had you asked me before starting your work, I would have told you then itself 😅

BTW, if you have some time and can work on this project, I can give you pointers on where to work and what to work. You can pick it up and work on it.

@evrins
Copy link
Collaborator Author

evrins commented May 21, 2024

@varbhat next time I will open a issue before making any changes
by now I have some time to work on this project
maybe you can assign me some task

@varbhat
Copy link
Owner

varbhat commented May 21, 2024

  1. Remove Tailwind CSS: You can use this Stylesheet here as a starting point: https://github.com/varbhat/varbhat.github.io/blob/main/layouts/partials/style.html
  2. Update to Svelte 5.0 when it's released
  3. Better RPC (JSON-RPC?) or Let's use REST with Server Side Events ( if it's Swagger compatible, then it's plus)
  4. Database Abstraction: I would prefer this:
    - pgx for Postgres. Maybe use embedded-postgres for testing. Note that embedded-postgres costs us 10 Megabytes of Size. If you use embedded-postgres for testing, make sure that it doesn't bloat up the final binary. if it does, let's not use it.
    - https://github.com/go-llsqlite/crawshaw for sqlite. Maybe we can move into mattn-sqlite. crawshaw is used by anacrolix/torrent i guess and anacrolix maintains it now. You take a call on this.
  5. Pagination of Torrents (This is high priority)
  6. To do (4), maybe we need to store state of anacrolix/torrent in DBs. Check if we can do that.
  7. Swappable Torrent Backends: Let's also provide libtorrent (There must be go binding somewhere) as a backend too! Let user choose it based on the build flag
  8. Version as a build flag param. Update the CI/CD script to do it too.
  9. Fine grained User permissions (Maybe we can use casbin?)
  10. Pluggable Storage Abstraction: It would be better if exatorrent can store files directly to S3/Blob Storage/etc. anacrolix/torrent has storage interface to do this. I initially thought that rclone would be better for this. just mount the rclone and start exatorrent in the rclone's mounted directory. but then again, you take a call on this. lot of people are interested in having S3/Blob Storage/Minio interfaces.
  11. Integrate with Jackett for Torrent Search.
  12. Create seperate issues for all these.

These are the things I wish exatorrent has. I would be more than happy if you have time and can work on this.

@varbhat
Copy link
Owner

varbhat commented May 21, 2024

@evrins , For libtorrent backend if you are interested, maybe we can use https://github.com/ElementumOrg/libtorrent-go ? Or let's create bindings using SWIG.

@varbhat
Copy link
Owner

varbhat commented May 21, 2024

Continuing, these might be priorities as well.

  • Set Upload/Download Rate Limits per torrent
  • Set Global Upload/Download Rate Limits(Only available for Admin)
  • Resource usage monitor?

@varbhat varbhat deleted the feat/add_orm branch May 21, 2024 09:49
@evrins
Copy link
Collaborator Author

evrins commented May 21, 2024

Ok I will work on them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants