Embeddings Pipeline
sentence-transformers → Redis DB1 → Semantic Search
What & Why
Embeddings convert text into dense numerical vectors such that similar meanings land near each other in vector space. This enables semantic search — finding a document about "invoice payment terms" when the user queries "when is the bill due?" — something exact keyword search cannot do.
Three principles make the Octopus approach effective:
Single model, consistent space. The same all-MiniLM-L6-v2 model is used for both storing vectors at ingestion time and encoding queries at search time. Because the model is deterministic, every vector lives in the same 384-dimensional coordinate space, so cosine similarities are meaningful.
CPU-only inference. No GPU infrastructure is required. The model loads once at process startup and serves all encoding requests from RAM. Inference takes 10–50 ms per text on a standard CPU, and the batched encoding path used during document ingestion processes many chunks in parallel, keeping ingest time low.
Shared storage layer. All three vector types — document chunks, MCP tool descriptions, and graph node descriptions — land in Redis DB 1 via Sagittarius. A single query-time scan can therefore cross-reference any combination of these sources.
Vector Storage Backends
Two backends are available. Sagittarius selects between them based on collection size and deployment context.
Selection rule: Sagittarius chooses sqlite-vec for small, in-process collections (development, testing, single-tenant deployments with limited data) and FalkorDB for production multi-tenant environments where vector counts scale into the millions. Both backends expose an identical search interface — switching backends requires only a configuration change.
Cosine Similarity
Vectors are compared using cosine similarity: the cosine of the angle between two vectors in 384-dimensional space. It measures direction, not magnitude, making it robust to differences in text length.
cos(θ) = A·B / (|A||B|)
A·B = dot product of vectors
|A|, |B| = vector magnitudes
Reference
Configuration
| Environment Variable | Default | Description |
|---|---|---|
EMBEDDING_MODEL | all-MiniLM-L6-v2 | sentence-transformers model name loaded at startup |
EMBEDDING_BATCH_SIZE | 32 | Number of texts encoded per batch during document ingestion |
Dimensions & Thresholds
| Property | Value |
|---|---|
| Vector dimensions | 384 |
| Vector dtype | float32 |
| Recommended similarity threshold | 0.7+ |
| Inference latency (CPU, single text) | 10–50 ms |
| Inference latency (batched ingest) | varies; batches of 32 default |
Python Usage
from sentence_transformers import SentenceTransformer
# Load once at startup (cached in memory)
model = SentenceTransformer('all-MiniLM-L6-v2')
# Encode a single string
vec = model.encode("What is the invoice due date?")
# vec.shape → (384,) dtype=float32
# Encode a batch (faster for many documents)
vecs = model.encode(["text one", "text two", "text three"])
# vecs.shape → (3, 384)
Where Each Vector Type Is Stored
| Vector type | Generated by | Redis key pattern | Backend |
|---|---|---|---|
| Document chunks | GraphRAG ingestion pipeline | doc:chunk:<id> | FalkorDB (prod) / sqlite-vec (dev) |
| MCP tool descriptions | Startup registration | mcp:tool:<name> | sqlite-vec |
| Graph node descriptions | Sagittarius entity indexer | graph:node:<id> | FalkorDB |