Improve RAG retrieval with hybrid search and reranking

“What does error E-417 mean?” contains an exact identifier. A semantically similar procedure for E-471 may be entirely wrong. Exact strings and appropriate text analysis matter here.

“How do I restore the device after an interruption?” may refer to the same procedure without using its terminology. Semantic retrieval can bridge wording differences, but cannot guarantee that every similar document concerns the correct device.

Azure documents combining full-text and vector queries. This is a supported product pattern rather than proof of improvement for every collection. Azure AI Search: hybrid search.

A hybrid RAG search runs through these stages:

1. Determine authorized source scope and relevant filters first.
2. Text and vector retrieval then produce candidates.
3. A fusion stage creates a combined ranking.
4. An optional additional model evaluates those candidates more closely against the question.
5. Context assembly subsequently decides which passages reach the answer model. It can remove duplicates, add necessary neighboring evidence and allocate the context budget.

Reranking and context assembly are therefore different jobs.

Record intermediate results during evaluation. If only the final answer is visible, it is difficult to tell whether the correct source was never found or later discarded. A small inspectable candidate trace can be more useful than another complex search component.

Full-text retrieval evaluates matches between query terms and document content. BM25 is a widely used method. Exact labels, uncommon terminology and identifiers are typical reasons to retain lexical retrieval.

Actual text analysis affects behavior. Capitalization, hyphens, word forms and abbreviations may be treated differently. Test real product codes and domain terms rather than assuming the algorithm name establishes correctness.

A glossary can connect company synonyms, but needs subject ownership. Similar product names are not automatically interchangeable. Overly broad replacement can reduce the precision text retrieval was intended to provide.

Vector search compares numerical representations of questions and content. It can find passages with similar meaning despite little shared vocabulary.

Similarity does not establish applicability. A passage for an older model may be extremely similar while containing the wrong maintenance rule. Product, version, location and effective-date metadata still matter.

Include the embedding model in the experiment record. Changing it can alter rankings and require reprocessing. Do not compare old document vectors with a new query representation without verifying compatibility.

Text and vector scores do not automatically share meaning or scale. Simply adding them can unintentionally favor one retrieval method. Rank-based fusion avoids some of that problem by considering an item's position in multiple lists.

Azure documents Reciprocal Rank Fusion for hybrid queries. It is one concrete fusion method; other products may expose different combinations and controls. Verify the implementation rather than treating a broad label as an exact configuration. Azure: hybrid queries.

Tune weights using a development collection. Repeatedly changing them until a few favorite questions look good can worsen other task types. Preserve an unchanged evaluation set for the final comparison.

A reranker compares the question with retrieved candidates more closely and can promote a relevant passage. It does not automatically search the complete corpus again. Evidence absent from the candidates remains absent after reordering.

Microsoft explicitly describes its semantic ranker as an additional evaluation of an existing result set. Product-specific limits are version-dependent rather than universal RAG rules. Azure AI Search: semantic ranking.

Also distinguish extracted search answers from generated answers. A search service may highlight useful source text; the subsequent language model may compose a new explanation. These outputs require different checks.

A fictional service manual contains the correct procedure for E-417. Text retrieval finds the identifier accurately. Another document describes general startup problems in accessible wording and ranks highly in vector retrieval.

The question combines both signals: “E-417 after a power outage: what next?” The fused list should contain the precise error procedure. Reranking may then prefer the passage that also addresses the outage context.

Before answering, the application still checks device type and applicable version. A mandatory safety prerequisite must remain in context. This is a proposed selection behavior, not a measured improvement for a particular product.

Permissions and validity filters change the eligible candidates. A search configuration that performs well without filters may retrieve less useful evidence in real operation. Evaluate under the restrictions actually deployed.

A filter error can resemble an algorithm failure. If the correct location is excluded, another reranker will not help. Establish that required evidence belongs in the permitted search space first.

Do not weaken authorization to improve retrieval scores. Finding more evidence by including restricted documents is not an improvement. The SharePoint guide explains why the access path needs separate acceptance.

A useful retrieval experiment can compare four variants:

• text retrieval
• vector retrieval
• hybrid retrieval
• hybrid retrieval with reranking

Keep source versions, identities and business references consistent to isolate each stage's contribution.

Group the test questions for a retrieval comparison into these types:

• exact identifiers
• paraphrased technical requests
• multilingual wording
• rules with exceptions
• questions with no available answer

Mark the required evidence for each case and have a subject reviewer assess the output.

Record irrelevant candidates, added latency and cost alongside successful retrieval. A stage that improves a few difficult cases may still be worthwhile without running on every easy question. Selective use requires clear rules and its own evaluation.

First determine whether required evidence appears in the candidate set actually used. Then inspect its ranking and whether it reaches the model. Only afterward judge the generated answer.

Microsoft distinguishes process and system evaluation in its RAG evaluators. That helps diagnose failures without replacing subject review of your tasks. Microsoft: RAG evaluators.

Strong retrieval does not prove a complete answer: the model may omit an exception. Conversely, it may produce a correct answer from general knowledge despite failed retrieval. A company application requiring evidence should detect that second case too.

Near-identical chunks can occupy the first several positions, displacing another necessary source from limited context. Inspect whether selected passages cover the claims needed to answer the question.

Coverage does not mean forcing several documents into every response. If one applicable procedure answers completely, extra material may confuse the model. Select for necessary evidence rather than an abstract diversity quota.

Handle old and new versions explicitly. Similar revisions presented together can encourage a blended answer. Effective-date filtering and clear version labels belong in selection rather than only in a final disclaimer.

More candidates and reranking can add processing. Measure complete response time across retrieval, ordering and generation. A fast search engine can still sit inside a slow overall workflow.

Include concurrent requests and difficult multi-part questions. Averages conceal outliers. Decide whether an outage permits a simpler previously evaluated search path or requires restricting the response.

Calculate cost per successfully completed task rather than per search request alone. Reduced rework can justify additional processing. Without measurable quality improvement, another stage mainly adds complexity.

A search score is not a universal probability that an answer is correct. Values may change across methods, models and index configurations. Avoid transferring a cutoff from a vendor example into production without checking its behavior on your collection.

Evaluate threshold effects on both missed answers and unsupported answers. Raising a cutoff may reduce irrelevant context while rejecting legitimate unusual wording. Lowering it may improve recall while supplying misleading passages. Choose according to the task and retain a clarification or abstention path. Record this trade-off so that a later update does not silently shift the existing answer boundary.

Decision record. Document the selected combination, evaluation set and failure groups improved. Record remaining weaknesses as well. This makes later model or index changes easier to assess.

Where to look next. If the right source is repeatedly missing, return to ingestion, chunking and filters. If it is present but poorly placed, investigate fusion and reranking. The chunking guide and RAG acceptance plan connect those decisions to concrete evidence.

Simple baseline. Preserve a simple baseline after selecting a more elaborate pipeline. It provides a useful reference when future changes increase latency or introduce regressions. A sophisticated stack should continue earning its place through observable benefits rather than becoming difficult to remove simply because it was deployed first.

Fixed source collection. Keep the source collection fixed during threshold tuning. If a document changes or a permission is corrected between runs, the score difference may not come from the setting under evaluation. Reproduce the earlier state where appropriate or rerun all compared variants against the new approved collection. This avoids selecting a configuration on an apparent gain caused by unrelated source maintenance.

These modernagents guides cover related decisions about AI knowledge assistants:

• AI knowledge assistants: sources, access and answers
• Split RAG content without losing its meaning
• Choose a vector database for RAG by operating needs
• Test a RAG knowledge assistant: sources, access and answers

For a workflow review, start with the task, systems involved and typical exceptions. Discuss it with modernagents.