A vector database stores text, images or audio as lists of numbers called embeddings, then returns the stored items whose numbers sit closest to the numbers of your question. That is how an AI chatbot finds the right paragraph in a thousand pages. For most projects it is a feature you switch on inside Postgres, not a separate product.
A vector database is a store built around one operation: given a vector, return the stored vectors nearest to it, quickly. Metadata filters, approximate indexes, sharding and replication all exist to keep that one operation usable as your data grows. It is a search index with a database wrapped around it, not a replacement for the database holding your orders and invoices.
This guide is the storage half of retrieval. What happens once those passages reach the model is in my RAG chatbot guide.
What are you actually storing in a vector database?
You are storing one fixed-length list of numbers per chunk of content, plus the ordinary fields you want to filter on. OpenAI's embeddings guide, read on 14 November 2026, puts it as plainly as anyone: "An embedding is a vector (list) of floating point numbers. The distance between two vectors measures their relatedness."
The length is fixed by the model, not by you. That page gives 1536 numbers for text-embedding-3-small and 3072 for text-embedding-3-large, with a dimensions parameter to shorten them. Every vector in a collection must be the same length, because the database compares them position by position.
Six words you will meet in every vendor's documentation:
| Term | What it means in practice |
|---|---|
| Embedding | A chunk of content turned into numbers by a model. Same model for storing and for searching, always. |
| Dimensions | How many numbers are in the list. Fixed by the model. |
| Distance | The arithmetic that says how far apart two lists are. Smaller means more alike. |
| Index | A structure that lets the database skip most rows instead of comparing all of them. |
| Recall | The share of the genuinely closest items the index returned. Below 100 percent means it missed some. |
| Metadata | The ordinary columns beside the vector: tenant, language, product, date. What you filter on. |
Nothing here is trained on your content. The embedding model is called as it comes, and what the database stores is its output.
How does similarity search find the right passage?
It measures the distance between your question's vector and every candidate vector, sorts by that distance and keeps the top few. The distance function is a choice you make once per column. The pgvector README, read on 14 November 2026, lists six: L2, inner product, cosine, L1, and Hamming and Jaccard for binary vectors.
Cosine distance is the usual pick for text, because it compares direction and ignores length. The same page adds that if vectors are already normalized to length 1, as OpenAI's are, inner product is the faster choice.
With no index, the database compares your query against every row. That is exact search, and pgvector says it "provides perfect recall". It is linear, so it slows in step with your row count. On a few tens of thousands of chunks that is usually fine, and I have shipped chatbots that never needed more.
What does an approximate index like HNSW cost you?
It costs you recall, and the amount is a setting rather than a surprise. HNSW stands for Hierarchical Navigable Small World, from a 2016 paper by Yu. A. Malkov and D. A. Yashunin, last revised in 2018. It builds a stack of graph layers, sparse at the top and dense at the bottom, and a search drops down the layers getting closer at each step.
pgvector's README gives the honest warning: an approximate index "trades some recall for speed", and "unlike typical indexes, you will see different results for queries after adding an approximate index". Test suites start failing for reasons that are not bugs.
Two numbers control the trade. One is set when the index is built and cannot be changed without a rebuild; the other is set per query. Defaults are not the same everywhere, so an index that felt accurate on one engine can feel sloppy on another. The values below come from the pgvector, Qdrant, Chroma and Weaviate references, read on 14 November 2026.
| Engine | Build setting and default | Query setting and default |
|---|---|---|
| pgvector | ef_construction 64, m 16 | hnsw.ef_search 40 |
| Qdrant | ef_construct 100, m 16 | ef, defaults to ef_construct |
| Chroma (single node) | ef_construction 100, max_neighbors 16 | ef_search 100 |
| Weaviate | efConstruction 128, maxConnections 32 | ef, dynamic by default |
Memory is the other bill. Weaviate's vector index page calls HNSW an in-memory index and sizes a node at 4 bytes per dimension, which it puts at 2GB to 12GB for a million vectors and 200GB to 1200GB for a hundred million, with edges on top. That is why people move off one server, and why quantization exists.
How do metadata filters change the result you get back?
They can quietly empty your result set, and this is the failure I see most often in projects that worked in testing. An approximate index walks a graph that knows nothing about your tenant_id, so a filter applied after the walk throws away most of what the index found.
pgvector states the arithmetic directly: if a condition matches 10 percent of rows, with HNSW and the default hnsw.ef_search of 40, "only 4 rows will match on average". Its answer since 0.8.0 is iterative index scans, SET hnsw.iterative_scan = strict_order or relaxed_order, which scan until enough rows are found. For a few distinct values it suggests partial indexes, and for many, partitioning.
Qdrant took a different route. Its indexing documentation describes a filterable HNSW index that adds extra edges from indexed payload fields, so the filter applies during the walk, not after. The catch is ordering: those edges are only built after the payload index exists, so adding one to loaded data means rebuilding the HNSW index. Version 1.16.0 added the ACORN algorithm for when two strict filters together still split the graph.
Pinecone constrains the data instead. Its index data overview requires metadata to be a flat JSON object: no nested objects, no null values, no keys starting with $. Tenants are separated by namespaces, which the same page puts at 100 per index on the Starter plan up to 1,000,000 on Enterprise.
Is Postgres with pgvector or SQLite enough for your project?
For most business chatbots and internal search tools, yes, and I start there unless the brief rules it out. pgvector adds a vector column type to Postgres 13 and newer, so you keep, in its own words, "ACID compliance, point-in-time recovery, JOINs, and all of the other great features of Postgres". One backup, one connection pool, one place where deleting a customer deletes their vectors too.
The limits are published: the vector type indexes up to 2,000 dimensions and halfvec up to 4,000, which covers the common models. Scaling is Postgres scaling: a bigger machine, read replicas or sharding.
SQLite has an answer too. sqlite-vec stores float, int8 and binary vectors in vec0 virtual tables with metadata, auxiliary and partition key columns, in plain C with no dependencies. Its README is blunt that it is pre-v1 and to expect breaking changes, so I reach for it on a desktop tool or a prototype first.
In words: no Postgres behind the app means start embedded, with Chroma or SQLite. With Postgres and under a few million vectors, stay there, especially when a write must change a row and its vector in one transaction. Go dedicated when the index outgrows one machine, or when you need high query rates with strict filters.
Which vector database fits, and what does each one give you?
License and hosting decide more than feature lists do, because they decide who is on call when it breaks. The table below was read from each project's own repository or documentation on 14 November 2026. Prices are left out on purpose: the only honest figure is the one on the vendor's pricing page today.
| Option | License | How you run it | Worth knowing |
|---|---|---|---|
| pgvector | PostgreSQL License | Extension in your own Postgres; preinstalled by many hosted providers | HNSW and IVFFlat indexes, exact search by default, vectors join your tables |
| Qdrant | Apache 2.0 | Self-host, or Qdrant Managed Cloud, Hybrid Cloud or Private Cloud | Filterable HNSW, plus the ACORN algorithm for strict filters |
| Weaviate | BSD 3-Clause, except the wl directory, which needs an enterprise license | Self-host, or managed Weaviate Cloud | HNSW, flat, dynamic and HFresh indexes; dynamic switches to HNSW past 10,000 objects |
| Milvus | Apache 2.0, under the LF AI and Data Foundation | Milvus Lite by pip, Standalone, distributed on Kubernetes, or Zilliz Cloud | HNSW, IVF, FLAT, SCANN and DiskANN indexes, GPU indexing, tenancy by database, collection or partition |
| Chroma | Apache 2.0 | Single node inside your app, or distributed and Chroma Cloud | HNSW on a single node, SPANN when distributed; the shortest path to a working index |
| Pinecone | Commercial, not open source | Pinecone-managed, or Bring Your Own Cloud in your AWS, GCP or Azure account | One schema combines dense, sparse and BM25 text; namespaces for tenancy |
Open source and self-hosted are not the same question. Qdrant, Milvus and Chroma ship under Apache 2.0, and Weaviate's core is BSD 3-Clause with its enterprise features under a separate license. All four also sell a managed service, so you can keep the exit option without running servers.
How do you test recall before customers see it?
You compare the index against exact search on your own data, because published benchmarks were not run on your documents. pgvector's monitoring section gives the cheapest method: run the query twice, once normally and once with the index off.
BEGIN;
SET LOCAL enable_indexscan = off; -- exact search
SELECT id FROM items ORDER BY embedding <=> '[...]' LIMIT 10;
COMMIT;
Run both lists for a few hundred real questions and count the overlap. That is your recall. Work through this before launch:
- Write 50 to 200 real questions with the passage that should answer each.
- Measure recall with no index first, so you know whether retrieval or chunking is weak.
- Measure again with every filter your app applies, not with filters off.
- Raise the query-time setting,
ef_searchoref, until recall is acceptable, then check latency. - Record the embedding model and dimension count next to the index, so nobody changes it by accident.
- Re-run the set after any reindex, upgrade or model change.
What goes wrong with a vector database in production?
Mostly the things that have nothing to do with search quality on day one.
- Changing the embedding model. Vectors from two models cannot be compared. Switching means re-embedding everything and paying that bill again.
- Reindexing is expensive. Qdrant's documentation calls rebuilding an HNSW index resource-intensive and says to avoid it where possible. Plan filters and payload indexes before you load data.
- Memory creep. An in-memory graph index grows with your row count whether or not traffic grows. Watch it as a capacity line.
- Shared indexes between tenants. pgvector warns that sharing an approximate index between tenants lets one tenant's vectors affect recall and speed for the others, and suggests list partitioning or separate tables.
- Deletes that linger. pgvector notes that vacuuming HNSW indexes can take a while, and suggests reindexing first.
- Exact strings. Pinecone's documentation puts it plainly: "Semantic search can miss exact keyword matches, while keyword search can miss semantically related results." Run both rather than arguing with one.
- A second datastore. It is a second thing to back up, monitor, patch and hand over. That is why my default answer is the database the app already has.
What else do people ask about vector databases?
What is a vector database in simple terms?
It is a filing system that files things by meaning instead of by name. Each item is stored as a list of numbers produced by a model, and a search returns the items whose numbers sit closest to your query's. Nothing in it understands your content; it measures closeness.
Do I need a vector database for RAG?
You need somewhere to put the embeddings, which is not the same as needing a new product. For a few thousand chunks, a vector column in the database you already run will do. A dedicated store earns its place when the index outgrows one machine or the query rate is high.
Which open-source vector database is best for RAG?
There is no single answer, and anyone who gives you one is selling something. Chroma is the quickest to start, Milvus is built for the distributed end, and Qdrant's filterable HNSW matters most when every search carries a filter. Pick on license, hosting and filtering behavior, then measure recall on your own data.
How is a vector database different from a relational database?
A relational database answers "which rows match these conditions" exactly. A vector database answers "which rows are most like this one" approximately, and returns a ranked list rather than a yes or no. They are not rivals; pgvector exists so an application can have both in one place.
How many vectors before Postgres stops being enough?
No vendor publishes a threshold, because it turns on your dimension count, your filters, your query rate and the memory on the box. The few million vectors in my decision flow is a rule of thumb for when to start measuring, not a published limit. The real signal is whether the HNSW index still fits comfortably in memory alongside everything else Postgres is doing.
I build AI chatbots and AI agents, and I founded MagizAI, my own AI chatbot product, so I am not a neutral voice here. If you want the storage layer chosen and measured properly, that is AI chatbot development work, or you can hire an AI developer hourly, with a free scoping call of about 30 minutes and a written quote within one business day.
Last reviewed 14 November 2026. Disclosure: I sell AI chatbot and AI agent development, so I am not neutral. I am CTO at ElySpace, a freelance developer for Marmaids LLC, and founder of WHMCSPilot and MagizAI, an AI chatbot and live chat product that uses retrieval of the kind described here. Licenses, defaults, limits and index behavior come from the pgvector, Qdrant, Weaviate, Milvus, Chroma, Pinecone, sqlite-vec and OpenAI pages linked above, all read on 14 November 2026.