Skip to content

Latest commit

 

History

History
113 lines (82 loc) · 3.58 KB

README.md

File metadata and controls

113 lines (82 loc) · 3.58 KB

LLM Memory: Persistent Memory for Language Models with Spice

Spice provides persistent memory capabilities for language models, enabling them to retain key information from conversations across sessions. This feature supports building more context-aware and intelligent applications by maintaining memory continuity.

Watch the Spice.ai LLM memory demo

Prerequisites

Ensure the following before starting:

  • Spice CLI installed.
  • Working directory is llm-memory: 
    git clone https://github.com/spiceai/cookbook.git
cd cookbook/llm-memory
  • The following environment variables set:
    • SPICE_OPENAI_API_KEY

Using LLM Memory

Step 1. Run Spice runtime

spice run
2025/01/27 11:29:49 INFO Checking for latest Spice runtime release...
2025/01/27 11:29:50 INFO Spice.ai runtime starting...
2025-01-27T19:29:50.856594Z  INFO runtime::init::dataset: Initializing dataset llm_memory
2025-01-27T19:29:50.857758Z  INFO runtime::init::model: Loading model [chat_model] from openai:gpt-4o...
2025-01-27T19:29:50.858938Z  INFO runtime::flight: Spice Runtime Flight listening on 127.0.0.1:50051
2025-01-27T19:29:50.859020Z  INFO runtime::metrics_server: Spice Runtime Metrics listening on 127.0.0.1:9090
2025-01-27T19:29:50.859210Z  INFO runtime::init::dataset: Dataset llm_memory registered (memory:store).
2025-01-27T19:29:50.865516Z  INFO runtime::http: Spice Runtime HTTP listening on 127.0.0.1:8090
2025-01-27T19:29:50.869294Z  INFO runtime::opentelemetry: Spice Runtime OpenTelemetry listening on 127.0.0.1:50052
2025-01-27T19:29:51.058851Z  INFO runtime::init::results_cache: Initialized results cache; max size: 128.00 MiB, item ttl: 1s
2025-01-27T19:29:52.248966Z  INFO runtime::init::model: Model [chat_model] deployed, ready for inferencing

Step 3. Start a chat session

spice chat

Step 4. Interact with the model

>>> spice chat

chat> Hi, my name is Alice and I work as a software engineer
Hi Alice! It's nice to meet you. How can I assist you today?

chat>  I live in Seattle. Tell me a joke about it
Sure, here's a Seattle-themed joke for you:

Why don't Seattle folks get lost in the woods?

Because they always follow the trail of coffee cups back home! ☕🌲

Hope that gives you a chuckle! Let me know if there's anything else you'd like to know or chat about.

Step 5. Check stored memories

spice sql

Then:

SELECT id, value FROM llm_memory;

Output:

+--------------------------------------+-------------------------------------+
| id                                   | value                               |
+--------------------------------------+-------------------------------------+
| 019319e4-ca14-7a12-a91a-f2c73528d304 | User's name is Alice                |
| 019319e4-ca14-7a12-a91a-f2d52fb70fba | Alice is a software engineer        |
| 019319e4-ca14-7a12-a91a-f2e2656ff222 | Alice lives in Seattle              |
+--------------------------------------+-------------------------------------+

Using Memory Tools Directly

Step 1. Store a memory directly

curl -XPOST http://127.0.0.1:8090/v1/tool/store_memory -d '{"thoughts": ["Alice deserves a promotion"]}'

Step 2. Load stored memories

curl -XPOST http://127.0.0.1:8090/v1/tool/load_memory -d '{"last": "10m"}'

Output:

[
  "Users name is Alice",
  "Alice is a software engineer",
  "Alice lives in Seattle",
  "Alice thinks she deserves a promotion"
]