FAISS
Master FAISS (Facebook AI Similarity Search): vector indexing, similarity search optimization, and production deployment patterns.
What is FAISS?
FAISS, short for Facebook AI Similarity Search, is a library for nearest-neighbor search and clustering over fixed-width vectors. It provides exact and approximate index structures, vector compression, CPU and GPU implementations, and composable index factories for building search pipelines.
FAISS is not a complete vector database. It does not by itself provide durable replication, tenant authorization, metadata truth, distributed consensus, online schema migration, or managed backup and recovery. A production system must wrap the library with those contracts.
The invariant to remember
Every query vector and indexed vector must share one declared vector-space contract, and every returned ID must resolve against the matching metadata generation. Fast neighbors are wrong when embedding model, dimensions, metric, normalization, IDs, or metadata drift apart.
Review vector databases if embeddings, distance metrics, and approximate nearest-neighbor search are new.
Follow one similarity query
1 Embedding
Create the query vector
Use the exact embedding model, preprocessing, dimensions, and normalization contract recorded by the index generation.
2 Admission
Choose search scope
Authenticate the caller, enforce tenant and resource policy, and resolve metadata filters or permitted shards before vector search.
3 FAISS
Traverse the index
Run exact scan, IVF cell probes, graph traversal, compressed-code comparison, or a composed factory pipeline.
4 Results
Refine and hydrate
Optionally re-rank candidates with full vectors, then map stable IDs to metadata from the same published generation.
5 Evidence
Observe quality
Record index version, parameters, filters, candidate count, latency, failures, and sampled recall against known ground truth.
Start with the exact baseline
Flat indexes compare the query with every stored vector. They require no training and establish exact neighbors for the chosen metric. That makes Flat useful for small corpora, low query volume, and the ground-truth sample used to evaluate every approximate candidate.
For cosine similarity, a common contract normalizes vectors and uses inner product. The normalization choice must be identical for index and query vectors; changing only one side changes the meaning of distance.
N x d
Flat distance work
An exact query compares N vectors across d dimensions before batching and hardware acceleration.
4 x d
float32 bytes/vector
Raw float32 storage is four bytes per dimension before IDs, graphs, centroids, or other index overhead.
recall@K
Approximation evidence
Compare the approximate top K with exact Flat neighbors over representative query slices.
p99
Serving target
Measure tail latency under the real concurrency, batch, filter, and update workload.
Choose an index family by constraint
Exact scan
Flat
Guarantees exact neighbors for L2 or inner product. Use when exactness is required, the corpus is manageable, or search volume cannot amortize a complex build.
Probe selected cells
IVF
Trains a coarse partition and scans vectors in selected inverted lists. nprobe raises recall and search work by visiting more cells.
Traverse a graph
HNSW
Uses a proximity graph with construction and search breadth controls. It can be fast and accurate but adds link memory and has different update constraints.
Compress vectors
Product Quantization
Encodes vector subspaces into compact lossy codes. It reduces memory and bandwidth while making code size, training data, recall, and optional re-ranking explicit trade-offs.
Composite factory strings can add preprocessing, IVF, encoding, fast scan, and refinement. Treat the full factory string and every trained artifact as release configuration, not an informal tuning note.
Reject poor candidates before an expensive build
Memory is often the first hard constraint because FAISS indexes are normally searched from RAM. Exact raw vector memory is only the starting point: add IDs, centroids, graph links, codebooks, temporary build memory, replicas, rollout overlap, and process headroom.
Trade exactness, memory, and search work explicitly
Choose an index family and change dataset, vector, breadth, compression, traffic, and RAM assumptions. The estimates identify a benchmark candidate; measured recall and latency remain the release evidence.
Candidate verdict
Estimated distance work exceeds the search envelope
Change index family, shard size, search breadth, compression, or traffic admission before loading the full production corpus.
30.9GB
24% of RAM budget
28.6GB
float32 values before index overhead
1.2M
Planning estimate per query
78.6%
154% modeled compute pressure
Ground truth
Use Flat exact search on a representative query and corpus sample to establish true neighbors and recall@K.
Sweep
Tune nprobe, efSearch, code size, re-ranking, threads, and batch size under the real traffic and filter mix.
Publish
Bind embeddings, metric, normalization, factory string, trained state, IDs, metadata snapshot, and benchmark evidence to one version.
This model intentionally excludes cache behavior, SIMD and GPU kernels, training quality, thread topology, filter selectivity, and storage I/O. Use it to reject obviously poor candidates, not to claim production latency.
The planner narrows the experiment. Only a benchmark on representative embeddings, queries, filters, hardware, threads, and traffic can approve an index.
Tune against a measured quality frontier
Sweep the number of coarse cells and nprobe. Train on a representative sample, inspect list imbalance, and include filter selectivity because filtering can leave too few eligible candidates.
Sweep graph degree, construction effort, and efSearch. Measure memory and build time as well as recall and latency; higher search breadth is not free.
Sweep code size, subquantizers, bits, residual encoding, optional OPQ, fast-scan variants, and exact or compressed refinement. Validate that dimensions satisfy the chosen encoding.
Sweep threads, batches, replicas, CPU or GPU placement, and admission. Benchmark p50 and p99 under concurrency rather than extrapolating one isolated query.
Keep the evaluation slices visible
- Head and tail entities, languages, locales, modalities, tenants, and time ranges
- Short, long, common, rare, adversarial, and out-of-distribution queries
- Filtered and unfiltered searches at realistic selectivity
- Fresh vectors, deleted entities, stale metadata, and missing hydration records
- Recall@K, precision or task outcome, latency, throughput, memory, build time, and cost
Treat filters and updates as system contracts
FAISS focuses on vector search. Metadata filtering, authorization, online mutation, deletion semantics, and result hydration often live outside the index and can dominate correctness.
Narrow first
Pre-filter
Resolve an eligible ID set or shard before vector search. This preserves policy but can fragment indexes or make small subsets inefficient.
Over-fetch
Search then filter
Retrieve extra vector candidates and discard ineligible results. It is simple, but selective filters can exhaust candidates and produce too few results.
Physical boundary
Partition by policy
Separate tenants, regions, or coarse categories into independent indexes. Isolation improves, while small shards and cross-partition ranking add cost.
Mutable overlay
Delta plus rebuild
Search a stable base and a small mutable delta, merge results, and rebuild generations periodically. Define duplicate IDs, tombstones, and cutover ordering.
Never return an unauthorized candidate and filter later at the client. Authorization belongs before result disclosure.
Publish the index as one immutable generation
An index generation includes more than a .faiss file. It binds the embedding model and preprocessing, dimensions, metric, normalization, factory string, trained parameters, vector IDs, deletion state, metadata snapshot, benchmark evidence, and artifact integrity.
Loading index release lab
Preparing release incidents...
Build the serving path around explicit ownership
FAISS is the search engine inside a larger service
Durability, policy, metadata truth, publication, and recovery remain outside the library boundary.
Vector contract
Embedding service
Produces versioned query and corpus vectors under one preprocessing, dimensions, metric, and normalization contract.
Offline generation
Index builder
Samples training data, trains when required, adds stable IDs, validates counts, benchmarks quality, and serializes immutable artifacts.
Durable publication
Artifact and manifest store
Stores trusted bytes, checksums, compatibility metadata, benchmark evidence, and the previous known-good generation.
Warm serving
Search replicas
Load and validate a complete generation, execute bounded queries, and expose versioned quality and systems telemetry.
Authorize and hydrate
Metadata and policy
Applies tenant and filter scope, resolves result IDs against matching metadata, and prevents unauthorized disclosure.
Plan sharding, replication, and recovery
Shard only after measuring one index
- By vector ID: balances corpus size but requires every query to fan out and merge top K.
- By tenant or policy: creates strong boundaries but can produce uneven and tiny indexes.
- By coarse semantic partition: reduces fan-out when routing is accurate, but routing mistakes lower recall.
- By hardware capacity: keeps each loaded index within RAM and build windows, while operational metadata must track ownership.
Every shard needs a generation ID, replica set, checksum, warmup state, query admission limit, and clear behavior when unavailable. A partial result must be distinguishable from a complete top K.
Rehearse these failures
- A replica fails during a traffic spike and remaining replicas approach memory or CPU saturation.
- A new generation loads successfully but has a recall or metadata-hydration regression.
- One shard is missing, slow, or on a different generation during top-K merge.
- The embedding service rolls forward before the matching corpus index is ready.
- A deletion or policy change must take effect before the next full rebuild.
Review the production checklist
Build these controls
- Establish exact Flat ground truth before tuning approximate candidates.
- Version the complete vector-space and metadata contract with every generation.
- Benchmark recall, latency, throughput, memory, build time, filters, and cost on representative slices.
- Load only trusted, integrity-checked artifacts and validate them outside the request path.
- Shadow and warm a complete generation before an atomic routing swap.
- Retain a known-good rollback generation and rehearse shard and model-version failures.
Avoid these traps
- Calling FAISS a database while durability, authorization, metadata, and recovery are unspecified.
- Comparing index families with invented accuracy or latency constants instead of measured ground truth.
- Mixing embedding models, dimensions, metrics, normalization, IDs, or metadata generations.
- Assuming one global
nprobeorefSearchvalue works across every query and filter slice. - Updating files in place, trusting mutable names, or loading serialized artifacts from untrusted sources.