Langchain Milvus - Handle switch between multiple databases

[tvvignesh]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

If I try to do operations in different databases of the vectorstore, Milvus, it doesn't work until I disconnect all existing connections. Otherwise, it always does the operation in the connection of 0th index of the global connections object

Only this works:

# This block gets called multiple times for different databases

connections.disconnect("default")
connections.remove_connection("default")

connections.disconnect("db1")
connections.remove_connection("db1")

vectorstore = Milvus(
            self.embeddings,
            collection_name=collection_name,
            auto_id=True,
            connection_args={
                "user": self.user,
                "password": self.password,
                "host": self.host,
                "port": self.port,
                "db_name": self.db_name,
            },
            index_params={}
        )

fs = LocalFileStore("./documents/kvdata/" + self.db_name + "_" + collection_name)
        store = create_kv_docstore(fs)
        parent_splitter = RecursiveCharacterTextSplitter(chunk_size=10000, chunk_overlap=2000)
        child_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=400)

        retriever = ParentDocumentRetriever(
            vectorstore=vectorstore,
            docstore=store,
            child_splitter=child_splitter,
            parent_splitter=parent_splitter,
        )

        retriever.add_documents(documents)

# Do subsequent operations with vectorstore

If I don't disconnect to all databases before doing all operations, all collections get created in the default milvus db and does not respect the db_name parameter in connection_args even when I noticed that the right db_name was being passed for the collection

Description

I have commented on the same issue here: milvus-io/pymilvus#2161 (comment) - this needs an urgent look since because of this, we are unable to use multiple databases with langchain_milvus without disconnecting all existing connections which is not the right way for us and is causing a lot of trouble.

I assume because of this code block:

if given_address is not None:
            for con in connections.list_connections():
                addr = connections.get_connection_addr(con[0])
                if (
                    con[1]
                    and ("address" in addr)
                    and (addr["address"] == given_address)
                    and ("user" in addr)
                    and (addr["user"] == tmp_user)
                ):
                    logger.debug("Using previous connection: %s", con[0])
                    return con[0]

While it gets the previous, connection, I am not sure if db_name parameter supplied in connection_args is respected when it reconnects.

System Info

langchain==0.2.10
langchain-community==0.2.9
langchain-core==0.2.22
langchain-experimental==0.0.62
langchain-huggingface==0.0.3
langchain-milvus==0.1.2
langchain-text-splitters==0.2.2

[tvvignesh]

@ccurme Got this working by making changes as mentioned in milvus-io/pymilvus#2161 (comment)

But this needs an update in langchain_milvus package as well as suggested

[dosubot]

Hi, @tvvignesh. I'm Dosu, and I'm helping the LangChain team manage their backlog. I'm marking this issue as stale.

Issue Summary

• The issue involves a bug in LangChain's integration with Milvus.
• Operations default to the first connection, ignoring the db_name parameter.
• This affects the use of multiple databases.
• You found a workaround using suggestions from a related Milvus issue.
• The LangChain Milvus package still needs an update to address the problem.

Next Steps

• Please let me know if this issue is still relevant to the latest version of LangChain by commenting here.
• If there is no further activity, this issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!