Idea: long-term memory for the agent across restarts

#1
by Corbenic - opened

I look at this AnythingLLM Space and I like that it is the real product image, with workspaces, local vector memory, and built-in Agents that can use a self-hosted model such as Ollama. When the local engine or the Space restart, those agent instructions and the workspace context must be fed to the model again from zero. Galahad is long-term memory for the agent, it can save and restore the KV cache byte-for-byte across restarts so the agent skip recomputation and keep memory between sessions. We also have faster first token on all 7 NVIDIA GPU types tested. Would you be open to try it, or want help integrating? https://github.com/corbenicai/galahad

Sietse

Sign up or log in to comment