Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify qa examples #70

Merged
merged 8 commits into from
May 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions .github/workflows/nb_runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,9 @@ jobs:
DEEPSEARCH_VERIFY_SSL: ${{ vars.DEEPSEARCH_VERIFY_SSL }}
DS_NB_PROJ_KEY: ${{ vars.DS_NB_PROJ_KEY }}
DS_NR_EXCLUDED: ${{ vars.DS_NR_EXCLUDED }}
DS_NB_SEM_ON_IDX_KEY: ${{ vars.DS_NB_SEM_ON_IDX_KEY }}
DS_NB_RAISE_ON_SEM_ERR: ${{ vars.DS_NB_RAISE_ON_SEM_ERR }}
DS_NB_SEM_ON_IDX_DOC_HASH: ${{ vars.DS_NB_SEM_ON_IDX_DOC_HASH }}
DS_NB_SEM_OFF_IDX_KEY: ${{ vars.DS_NB_SEM_OFF_IDX_KEY }}
DS_NB_SEM_OFF_IDX_DOC_HASH: ${{ vars.DS_NB_SEM_OFF_IDX_DOC_HASH }}
DS_NB_QA_IDX_KEY: ${{ vars.DS_NB_QA_IDX_KEY }}
DS_NB_QA_DOC_HASH: ${{ vars.DS_NB_QA_DOC_HASH }}
DS_NB_QUESTION: ${{ vars.DS_NB_QUESTION }}
DS_NB_GEN_TIMEOUT: ${{ vars.DS_NB_GEN_TIMEOUT }}
DS_NB_INDEX_KEY: ${{ vars.DS_NB_INDEX_KEY }}
run: poetry run python -m nbrunner.nb_runner
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ This section will showcase examples of semantic capabilitilies in the area of Q&

| | Name | Description |
| -- | ----------------- | ----------- |
| 1. | [QA on a specific document][qa_single_doc] | Interact with a specific document using Conversational AI <br /> [<img src=".readme_resources/documentqa.png" width="300px" />][qa_single_doc] |
| 2. | [QA on a document collection][qa_doc_collection] | Interact with a document collection using Conversational AI <br /> [<img src=".readme_resources/qa_doc_collection.png" width="300px" />][qa_doc_collection] |
| 1. | [QA quick start][qa_quick_start] | Get started with semantic ingestion, RAG, and retrieval.<br /> [<img src=".readme_resources/documentqa.png" width="300px" />][qa_quick_start] |
| 2. | [QA deep dive][qa_deep_dive] | Explore advanced RAG and semantic retrieval capabilities.<br /> [<img src=".readme_resources/qa_doc_collection.png" width="300px" />][qa_deep_dive] |


### Bring your own
Expand Down Expand Up @@ -144,7 +144,8 @@ For individual model usage, please refer to the model licenses found in the orig
[data_extract_figures]: ./examples/data_extract_figures/
[data_query_chemistry]: ./examples/data_query_chemistry/
[data_query_chemistry_patcid]: ./examples/data_query_chemistry_patcid/
[qa_single_doc]: ./examples/qa_single_doc
[qa_quick_start]: ./examples/qa/qa_quick_start.ipynb
[qa_deep_dive]: ./examples/qa/qa_deep_dive.ipynb
[qa_doc_collection]: ./examples/qa_doc_collection
[bring_your_own_pdf]: ./examples/bring_your_own_pdf/
[bring_your_own_converted_documents]: ./bring_your_own_converted_documents/
Expand Down
16 changes: 0 additions & 16 deletions dsnotebooks/settings.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from datetime import datetime
from typing import Optional

from deepsearch.cps.queries import ConstrainedWeight
from dotenv import find_dotenv
from pydantic.v1 import BaseSettings, validator

Expand Down Expand Up @@ -36,18 +35,3 @@ def set_kg_key(cls, v):
class CollOptionalNotebookSettings(NotebookSettings):
proj_key: Optional[str] = None
index_key: Optional[str] = None


class CollQANotebookSettings(ProjectNotebookSettings):
sem_on_idx_key: str
retr_k: int = 5
text_weight: ConstrainedWeight = 0.1
rerank: bool = False
skip_ingested_docs: bool = True
raise_on_sem_err: bool = True


class DocQANotebookSettings(CollQANotebookSettings):
sem_on_idx_doc_hash: str
sem_off_idx_key: str
sem_off_idx_doc_hash: str
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Document Collection QA
# Document QA

Deep Search allows users to interact with the documents using conversational AI, i.e. you interact with a virtual assistant which answers your questions using the information in a document collection.
Deep Search allows users to interact with the documents using conversational AI, i.e. the user interacts with a virtual assistant which answers their questions using the information from a document collection or a specific document.

:point_right: Run the [doc_collection_qa.ipynb](./doc_collection_qa.ipynb) notebook.
:point_right: For getting started, check out [QA Quick Start](./qa_quick_start.ipynb).

:point_right: For advanced usage, check out [QA Deep Dive](./qa_deep_dive.ipynb).


### Access required
Expand Down
Loading
Loading