Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add postgres support #28

Open
5 tasks
waqasbhatti opened this issue Jul 31, 2018 · 0 comments
Open
5 tasks

add postgres support #28

waqasbhatti opened this issue Jul 31, 2018 · 0 comments
Assignees
Labels
backend Issue dealing with the search/dataset backend infra enhancement New feature or request hard issue This will take some time to fix

Comments

@waqasbhatti
Copy link
Owner

This will involve:

  • separating dbsearch.py into two modules: sqlite.py and postgres.py
  • adding in support for kwargs for indexserver to pick up which backend to use
  • adding support for full-text search in postgres (it doesn't have BM25, but maybe pg_trgrm ops on a GIN index on the appropriate FTS columns is enough? we could also implement BM25 in Python and rerank after results come back)
  • searchserver_handlers will have to something like from ..backend import sqlite, postgres instead of from ..backend import dbsearch
  • decide what to do with the lcc-datasets.sqlite and lcc-index.sqlite databases we use (I think we can keep them around, no need to shove everything into postgres)

The translation between sqlite -> postgres:

  • separate collections go into separate tables instead of separate databases
  • we can now join across tables (if we ever wanted to)
@waqasbhatti waqasbhatti added enhancement New feature or request hard issue This will take some time to fix backend Issue dealing with the search/dataset backend infra labels Jul 31, 2018
@waqasbhatti waqasbhatti self-assigned this Jul 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Issue dealing with the search/dataset backend infra enhancement New feature or request hard issue This will take some time to fix
Projects
None yet
Development

No branches or pull requests

1 participant