Choose a vector database for RAG by operating needs

The store holds numerical representations and often metadata. It does not automatically create complete sources, valid permissions or correct answers. Extraction, chunking, approval and generation remain separate responsibilities.

A poorly prepared passage remains poorly prepared in a faster database. Missing product codes or exceptions cannot reliably be recreated by storage. Start with a clean reference collection and known required evidence.

The RAG architecture guide describes the complete flow. Derive the storage decision from that design instead of building the application around a product chosen in advance.

Record searchable-object count and size, vector dimensions, change frequency and expected concurrent requests. Original file count is insufficient: one manual may produce many chunks while a short form produces few.

List required filters, including tenant, user group, product, language, location and effective period. Record how selective they are. Searching almost the entire collection differs from searching a small authorized subset.

Also define acceptable outages and operating hours. An occasionally used internal assistant may need different availability from a central service application. These conditions affect architecture and cost more than a single-query demonstration.

The following table compares three architecture paths for vector storage in RAG by when to consider each and the responsibility it adds.

Capabilities overlap, so compare functions actually required. A long feature list is not an advantage when the application uses little of it and operations become harder.

Check incremental load on existing infrastructure. Sharing a platform can simplify administration, but large index builds must not disrupt important transactional work. A separate service isolates that load while introducing synchronization responsibilities.

pgvector adds vector similarity search to PostgreSQL and documents exact and approximate methods. Teams already operating PostgreSQL therefore have a reasonable baseline to evaluate. This does not establish superiority for every collection or workload. pgvector: official documentation.

Keeping metadata and vectors close together may simplify the design. Still inspect actual queries, index configuration and competition with other database tasks. A familiar database does not make a new retrieval workload free.

PostgreSQL also provides row security policies. Their effectiveness depends on roles and execution context; the feature alone does not prove correct preservation of external document permissions. PostgreSQL: row security policies.

A specialist store can suit particular retrieval or scaling needs. Qdrant, for example, documents filtering on stored properties and the importance of appropriate indexes for filter fields. Qdrant: filtering.

Providing a filter does not fully control what each user is allowed to access. The application must derive the correct restriction from authenticated identity and prevent users expanding it. Source deletions and permission changes must also reach the stored metadata.

Evaluate routine operations alongside queries: ingestion, backup, restoration, diagnosis and upgrades. A retrieval benchmark does not perform those jobs. The appropriate service is one whose complete responsibilities the team can manage or obtain reliably.

Managed search can combine text retrieval, vectors, filters and ranking. That may reduce integration work when those functions are required. It can also bind the application to specific query and index models.

Azure documents hybrid retrieval combining text and vector signals. Compare it using the same questions and filters as other options. Azure AI Search: hybrid search.

Review the particular deployment and current billing model. Managed offerings can have dedicated or usage-based variants. Avoid assuming that every service always charges the same fixed capacity or becomes entirely free when idle.

Exact search determines nearest items under the chosen distance over the considered collection. Approximate methods can reduce work while allowing differences in which neighbors are found. Practical impact depends on the data and task.

Use an inspectable reference for a bounded test collection to detect missed evidence. Maximum speed is of little value if a rare applicable procedure disappears consistently.

Distance itself is not business correctness. Even the exact nearest vector can be the wrong source. Evaluate technical neighbor retrieval and actual relevance separately.

The order of filtering and approximate retrieval affects outcomes. pgvector explains that filters with approximate indexes can apply after the index scan, leaving fewer matching rows; its documentation describes several mitigations. pgvector: filtering and iterative search.

Do not assume other products behave identically. Evaluate each option with both highly selective and broad filters. Unrestricted search across all documents does not represent a permission-sensitive assistant with many user groups.

When too few results remain, do not remove security restrictions. Investigate search parameters, index partitioning or another retrieval path. An appropriate empty result is preferable to filling the context with another team's content.

A synthetic example uses 100,000 vectors, 1,536 values per vector and four bytes per value. The raw values occupy 614,400,000 bytes, approximately 614 MB in decimal units. These assumptions are not a recommendation for a particular embedding model.

The actual database also needs index structures, metadata, text, management overhead and potentially replicas. Backups and parallel rebuilding add further capacity. Raw-vector arithmetic is one planning component rather than a complete storage order.

Check compression and its quality implications. Compare savings against retrieval behavior and operating effort. A vendor's published footprint is not a measurement of your collection.

Same conditions. Use the same sources, embeddings and filters. Changing the model or chunking at the same time prevents attribution to the store. Begin with a small number of genuinely suitable candidates.

Realistic cases. Include exact identifiers, semantic wording and small authorized subsets. Measure quality and latency at realistic concurrency. Add ingestion while requests continue so the evaluation covers more than a static ideal state.

Recorded configuration. Record instance size, configuration and software version. Without them, results are difficult to reproduce. The goal is a defensible decision for your workload rather than a universal leaderboard.

Failed and slow requests. Include failed and slow requests in the result, not only successful query averages. A service can have an attractive median while occasionally exceeding the application's usable response time. Decide which failure rate and response window the business can tolerate before comparing options.

A withdrawn source must be removable with its associated chunks. Verify that stable document identifiers support targeted cleanup and determine which stored answers or other derivatives also need treatment.

Changing the embedding model may change dimensions or vector space. Plan construction, evaluation and activation of the replacement collection. Parallel indexes temporarily require more capacity but can enable a controlled transition.

Retain original text and metadata so representations can be regenerated. Vectors alone are not a convenient interchange format for all company knowledge. Test exit from the product in practice.

Restore a small representative backup into a controlled environment. Confirm that source identities, metadata and search behavior survive. Then reconcile changes that occurred after the backup, especially revoked permissions and withdrawn documents.

For multiple organizations or departments, test that one scope cannot affect another's authorized result set. Performance isolation may matter as well: a large import for one collection should not unexpectedly make another team's essential search unusable. The appropriate boundary depends on the agreed operating mandate.

These tests reveal operational properties a query-only benchmark misses. They also establish whether your team has the access and documentation required to recover the service when the usual operator is unavailable. Record the measured restoration effort and compare it with the permitted outage window. A backup without a tested way back is not yet a dependable recovery plan.

Cost items. Include hosting or service usage, storage, backups, transfer, monitoring and human effort. Self-hosted components require upgrades and recovery; managed services still need configuration, source maintenance and cost control.

Assigned responsibility. Assign responsibility for slow index builds, failed deletions and unexpected usage. A low price without available operators can become expensive. Conversely, extensive capabilities do not justify recurring effort irrelevant to the task.

Selection comes last. The cost guide supports the calculation. Select only after evaluating retrieval, access, updates and restoration. The vector database then becomes a justified component of the assistant rather than a goal in itself.

Decision triggers. Record what would change the decision: substantially greater volume, a new permission model, additional languages or a tighter response target. This prevents both premature migration and indefinite attachment to a choice whose original assumptions no longer hold.

Handover check. Check the handover package with someone other than the original implementer. They should be able to identify active collections, verify the latest backup and explain how a withdrawn document is blocked. Missing access or undocumented dependencies are concrete operating costs. Resolve them before treating a successful prototype as a maintainable production choice.

These modernagents guides cover related decisions about AI knowledge assistants:

• AI knowledge assistants: sources, access and answers
• RAG architecture from source document to checked answer
• Improve RAG retrieval with hybrid search and reranking
• RAG costs: budget for setup, models and knowledge maintenance

For a workflow review, start with the task, systems involved and typical exceptions. Discuss it with modernagents.