Overview
Galahad is a memory layer for AI systems that preserves a model’s prior reading of text. It can restore that reading when the same material is needed again, reducing repeated token processing, and it stores documents as exact text rather than lossy embeddings.
Its C++ core works with llama.cpp, vLLM, and SGLang. The package supports Linux x86-64 with Python 3.10 through 3.14, and the core library can also be linked directly into C++ applications.
Key Features
- Taliesin saves model KV caches to disk and restores them for later requests.
- Blaise stores documents byte-for-byte and selects the chapter relevant to a question.
- Long texts are processed in parts, with saved readings moved between disk and GPU memory as needed.
- Persistent memory survives restarts and is encrypted at rest with a key that remains on the user’s machine.
- Agent connectors record execution steps, while replay identifies the exact step that changed between runs.
Integrations and Operation
For vLLM, Galahad operates as a KV connector. It integrates with SGLang as a HiCache storage backend and uses llama.cpp slot save-and-restore capabilities. In an embedded C++ setup, llama.cpp is built into the Galahad library and exposed through its C API.
The Python wheel includes the shared Galahad library. Installation is available through the galahad-kv package, with a diagnostic command for checking that the library loads, the licence is valid, and storage is writable.
Measured Results
The published measurements report a 99.6% token cache return rate and 14× faster performance on vLLM, with 0.59 seconds per question. For document retrieval, the project reports 100 correct answers out of 100 while reading 670 tokens rather than 9,700.
Tests with texts from one million to 50 million tokens reported constant GPU memory use of 34.1 GB. The embedded C++ configuration reports 100 out of 100 correct answers at 0.65 seconds per question.
Pricing
A free licence is available for one GPU under non-commercial terms. Activation requires organization and email details and acceptance of those terms.