Local RAG with our own embeddings
The engine now serves /v1/embeddings. Full local RAG — embed, index, retrieve, generate — on one binary, no cloud, no Python.
Milestone: fully local RAG on one binary — embed, index, retrieve and generate on your own hardware, no cloud, no Python. Full technical write-up → CODEBASE reference
Embeddings, in the engine
The engine's server now exposes /v1/embeddings alongside /v1/chat/completions. Load an embedding model — nomic-embed-text-v1-GGUF is one command away — and the same binary that generates tokens also produces 768-dim vectors. No second server, no Python, no cloud API.
This site's own semantic search runs on exactly this: every page is chunked, embedded, and indexed. The search box on 1bit.monster is the engine's embeddings in production.
The pipeline
# embed a corpus
curl http://127.0.0.1:8088/v1/embeddings \
-H 'Content-Type: application/json' \
-d '{"input": ["doc one", "doc two"], "model": "nomic-embed-text-v1-GGUF"}'
Normalize the vectors, store them (a SQLite table or a JSON file), and retrieval is a dot product. For 10,000 chunks it's milliseconds in any language; for a million, a vector index or an ANN library slots in behind the same API.
Why local matters
Your documents never leave the machine. A legal review, a codebase, medical notes, a personal vault — embed and retrieve them with an engine that runs entirely on your hardware. The privacy argument isn't a feature pitch, it's the default.
And it's fast: the embedding model runs on the same NPU/ROCm/CPU backends as generation. Batch embedding 57 chunks of this website took well under a minute on a mid-range machine.
RAG end to end
The pattern: embed the query with the same model, take the top-k chunks by cosine similarity, stuff them into the system prompt, generate. That's it. One binary, one port, three API calls. We ship the search half as this site's search.html, and the generation half is the same build/1bit you already run.