A vector database stores high-dimensional embeddings and answers nearest-neighbor queries fast. It is the storage substrate underneath most [[Retrieval-Augmented Generation]] systems — the component that decides, in milliseconds, which of your millions of chunks the model gets to read.
What you are actually buying
The core primitive is approximate nearest-neighbor (ANN) search. Exact search
over millions of vectors is too slow, so indexes like HNSW or IVF trade a sliver
of recall for orders-of-magnitude speed. The tuning knobs — ef_search, number
of probes, graph connectivity — are a direct recall-versus-latency dial. Knowing
where to set that dial for your workload is most of the job.
Choosing one
The market splits into three camps: purpose-built engines (Pinecone, Weaviate, Qdrant), extensions to existing databases (pgvector on Postgres), and libraries you embed yourself (FAISS). My default is pgvector until scale forces a dedicated engine — one fewer system to operate, transactional consistency for free, and joins against your relational data.
Beyond similarity
Pure vector similarity is rarely enough. Metadata filtering (tenant, recency, document type) and hybrid scoring against keyword search close the gap between "semantically close" and "actually relevant." Getting that filtering right is often the difference between a demo and a product.
Whatever you pick, treat index health as a first-class signal. Recall drift as the corpus grows is silent and corrosive, which is why I wire index metrics into [[LLM Observability]] from day one rather than waiting for users to complain.