Title: Lecture 3 — Choosing a Vector Database Without the Hype
Course: AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation > Section: Week 03 — Vector Database Selection: Qdrant, Milvus, Weaviate, and pgvector

📖 Lecture — Choosing a Vector Database Without the Hype

Every production RAG system needs somewhere to store embeddings and search them by similarity, and the vector database market has exploded with options. If you search for "best vector database" you'll find dozens of benchmark posts each crowning a different winner. The truth is more useful than a leaderboard: there is no single best vector database, only the best vector database for your constraints. This week we'll build a tradeoff framework around four real options — pgvector, Qdrant, Weaviate, and Milvus — so you can make a defensible choice instead of a trendy one.

Start with the question, not the tool

Before comparing benchmarks, ask three questions about your system: (1) How much data will you actually index — thousands, millions, or billions of vectors? (2) Do you need heavy metadata filtering alongside similarity search (e.g., "find similar chunks, but only from documents tagged 'finance' and updated this year")? (3) What's your team's operational appetite — do you already run Postgres, or are you comfortable operating a new distributed system? These three answers narrow the field faster than any benchmark chart.

The four contenders

pgvector is a Postgres extension, not a separate database. If your team already runs Postgres for application data, pgvector lets you store embeddings in the same database, in the same transactions, with the same backup and access-control tooling you already trust. That's the single biggest argument for it: zero new operational surface area. Enable it with:

CREATE EXTENSION vector;

It works cleanly with Postgres 16 and 17. The historical knock against pgvector was twofold: a lower practical dimension ceiling (commonly cited around 2000 dimensions for indexed columns, versus higher or unlimited caps in dedicated engines) and CPU-bound ingestion, since it doesn't have native GPU bulk-loading. Both are real constraints — but they matter far less than the old "Postgres extension = always slower" rule of thumb suggests. Recent benchmarks of pgvector paired with the pgvectorscale extension show hundreds of queries per second at roughly 99% recall on tens of millions of vectors, which is enough for the overwhelming majority of production RAG workloads. The honest guidance: pgvector is an excellent default for moderate-scale ingestion under roughly 10 million vectors, especially when you're already on Postgres. Qdrant is a Rust-based, purpose-built vector engine and currently the strongest default when you need low-latency search combined with rich metadata filtering. Its filtering is built into the core query planner rather than bolted on, which is why it tends to win on filtered-query latency. Getting it running locally takes one command:

docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant:latest

That exposes a REST API on port 6333 and gRPC on port 6334. On the client side, pip install "qdrant-client[fastembed]" gives you an embedding-ready Python client out of the box. Weaviate differentiates itself with built-in hybrid search (combining vector similarity with keyword/BM25 scoring in a single query) and a modular ecosystem of "vectorizer" modules that can generate embeddings for you as data is ingested. If your use case genuinely needs hybrid search or you want vectorization handled inside the database layer, Weaviate removes glue code you'd otherwise write yourself. Milvus (and its managed cousin, Zilliz Cloud) is built for billion-scale collections and supports GPU-accelerated indexing and search. If you're operating at a scale where CPU-bound ingestion becomes the bottleneck — think hundreds of millions to billions of vectors — Milvus's GPU acceleration is a real architectural advantage, not a marketing bullet point.

Grounding the choice in numbers

Benchmarks at 10 million vectors report p99 query latencies of roughly 12ms for Qdrant, 16ms for Weaviate, and 18ms for Milvus under filtered-search conditions — consistent with Qdrant's reputation for low-latency filtered retrieval. But note what these numbers do not tell you: they don't account for your team's operational cost of running and monitoring a new distributed system, and they say nothing about ingestion throughput at GPU scale, where Milvus pulls ahead.

Engine | Best for | p99 latency @ 10M vectors (filtered) | Ops overhead | Standout feature
pgvector | Teams already on Postgres, moderate scale (\<10M vectors) | Competitive with pgvectorscale (100s of QPS @ \~99% recall) | Lowest — reuses existing Postgres | Zero new infrastructure
Qdrant | Low-latency filtered RAG | \~12ms | Low–moderate (single Rust binary/container) | Query-planner-native filtering
Weaviate | Hybrid (vector + keyword) search | \~16ms | Moderate | Built-in vectorizer modules
Milvus / Zilliz | Billion-scale, GPU ingestion | \~18ms | Higher (distributed architecture) | GPU-accelerated indexing

Correcting two misconceptions

"A dedicated vector database is always faster and more scalable than pgvector, so never use Postgres." This was defensible advice a few years ago; it isn't anymore. Recent pgvector/pgvectorscale benchmarks reach hundreds of QPS at approximately 99% recall on tens of millions of vectors — solidly production-grade for most RAG applications. The correct rule: use pgvector when you're already on Postgres and your scale is moderate; reach for a dedicated engine only when you're approaching billion-scale, need GPU-accelerated bulk ingestion, or require built-in vectorization/hybrid search that pgvector doesn't offer. "Vector databases are interchangeable, so just pick whichever has the best benchmark QPS." Raw throughput is one input, not the whole decision. Purpose-built engines and Postgres extensions differ architecturally — vector-optimized storage engines and query planners with GPU bulk-ingestion paths, versus a CPU-bound index living inside a general-purpose relational database. Operational fit (do you want to run a new distributed system?), filtering requirements, and ecosystem (client libraries, managed hosting, vectorizer modules) matter just as much as a QPS number pulled from someone else's benchmark environment. The skill this week is building is judgment: given a set of constraints, can you name the engine that fits and explain why in terms of scale, filtering, and ops — not just repeat a benchmark chart from memory.