Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graph store with Anthropic LLM and Ollama embedder not working #2115

Open
acgonzales opened this issue Dec 28, 2024 · 1 comment
Open

Graph store with Anthropic LLM and Ollama embedder not working #2115

acgonzales opened this issue Dec 28, 2024 · 1 comment

Comments

@acgonzales
Copy link

acgonzales commented Dec 28, 2024

🐛 Describe the bug

I have this config and basic code to add a memory:

from mem0 import Memory

config = {
    "vector_store": {
        "provider": "qdrant",
        "config": {
            "collection_name": "test",
            "host": "localhost",
            "port": 6333,
            "embedding_model_dims": 768,
        },
    },
    "llm": {
        "provider": "anthropic",
        "config": {
            "model": "claude-3-5-sonnet-20241022",
            "temperature": 0.1,
            "max_tokens": 8192,
        },
    },
    "graph_store": {
        "provider": "neo4j",
        "config": {
            "url": "neo4j://localhost:7687",
            "username": "neo4j",
            "password": "password",
            "embedding_model_dims": 768,
        },
    },
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "nomic-embed-text:latest",
            "ollama_base_url": "http://localhost:11434",
        },
    },
    "version": "v1.1"
}

m = Memory.from_config(config)

res = m.add("I am working on improving my photography skills. Suggest some online courses.", user_id="john")
print(res)

The code above throws an error on the m.add statement.

anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'tool_choice: Input should be a valid dictionary or object to extract fields from'}}

It works well without the graph_store key in config.

I've been seeing examples where they only have graph_store in their config but I'm having issues with that as well and this is the error I'm getting.

config = {
    # "vector_store": {
    #     "provider": "qdrant",
    #     "config": {
    #         "collection_name": "test",
    #         "host": "localhost",
    #         "port": 6333,
    #         "embedding_model_dims": 768,
    #     },
    # },
    "llm": {
        "provider": "anthropic",
        "config": {
            "model": "claude-3-5-sonnet-20241022",
            "temperature": 0.1,
            "max_tokens": 8192,
        },
    },
    "graph_store": {
        "provider": "neo4j",
        "config": {
            "url": "neo4j://localhost:7687",
            "username": "neo4j",
            "password": "password",
            "embedding_model_dims": 768,
        },
    },
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "nomic-embed-text:latest",
            "ollama_base_url": "http://localhost:11434",
        },
    },
    "version": "v1.1"
}
ValueError: shapes (0,1536) and (768,) not aligned: 1536 (dim 1) != 768 (dim 0)

Versions

  • Mem0 = 0.1.36
  • Neo4j = 4.1.13
  • Anthropic (PIP) = 0.42.0
@kevinprice-se
Copy link

kevinprice-se commented Jan 6, 2025

I am also experiencing this issue.

It looks like when specifying a graph store the solution is not handling the request properly for tools and tool_choice for Anthropic

A reference to the mem0 code for constructing a request to Anthropic:

    def generate_response(
        self,
        messages: List[Dict[str, str]],
        response_format=None,
        tools: Optional[List[Dict]] = None,
        tool_choice: Dict[str, str] = {"type": "auto"},
    ):
        """
        Generate a response based on the given messages using Anthropic.

        Args:
            messages (list): List of message dicts containing 'role' and 'content'.
            response_format (str or object, optional): Format of the response. Defaults to "text".
            tools (list, optional): List of tools that the model can call. Defaults to None.
            tool_choice (str, optional): Tool choice method. Defaults to "auto".

        Returns:
            str: The generated response.
        """
        # Separate system message from other messages
        system_message = ""
        filtered_messages = []
        for message in messages:
            if message["role"] == "system":
                system_message = message["content"]
            else:
                filtered_messages.append(message)

        params = {
            "model": self.config.model,
            "messages": filtered_messages,
            "system": system_message,
            "temperature": self.config.temperature,
            "max_tokens": self.config.max_tokens,
            "top_p": self.config.top_p,
        }
        if tools:  # TODO: Remove tools if no issues found with new memory addition logic
            params["tools"] = tools
            params["tool_choice"] = tool_choice

        response = self.client.messages.create(**params)
        return response.content[0].text

I am still reviewing the code but it looks like tool_choice: str = "auto", is not defaulted correctly and I am not sure if the format of the tools provided aligns with what Anthropic expects

What is defined in the mem0 code


EXTRACT_ENTITIES_TOOL = {
    "type": "function",
    "function": {
        "name": "extract_entities",
        "description": "Extract entities and their types from the text.",
        "parameters": {
            "type": "object",
            "properties": {
                "entities": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "entity": {
                                "type": "string",
                                "description": "The name or identifier of the entity."
                            },
                            "entity_type": {
                                "type": "string",
                                "description": "The type or category of the entity."
                            }
                        },
                        "required": ["entity", "entity_type"],
                        "additionalProperties": False
                    },
                    "description": "An array of entities with their types."
                }
            },
            "required": ["entities"],
            "additionalProperties": False
        }
    }
}

What is defined in the Anthropic documentation
https://docs.anthropic.com/en/docs/build-with-claude/tool-use#example-simple-tool-definition

{
  "name": "get_weather",
  "description": "Get the current weather in a given location",
  "input_schema": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state, e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "The unit of temperature, either 'celsius' or 'fahrenheit'"
      }
    },
    "required": ["location"]
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants