Before jumping to complex architectures, you can first try tweaking standard parameters: changing the chunking size and overlap, adding metadata to help filter, swapping the embedding model, or changing your storage engine. If you try all of those basic adjustments and it still doesn't work, you can consider more complex strategies.
Here are 5 advanced search strategies to implement when simple RAG isn't enough:
Semantic search is the standard RAG strategy that matches chunks by meaning, not exact words.
It turns your question and the document chunks into "meaning fingerprints" to find matches even if the wording is completely different.
Use it when you know your users don't know the exact words used in the document, causing them to imagine their own words to search.
It is highly effective when concepts matter more than the exact string. For instance, a query for "data sharing rules" can easily find a clause titled "confidentiality and data handling".
Keyword search is a traditional method, but it is still highly useful. Not everything needs to be AI.
This strategy matches the actual words you typed without any interpretation.
You should use this when you need precise legal terms, product terms, exact codes, or SKUs.
If you know your users will use the correct terms, just use a keyword search.
For maximum accuracy, you need a hybrid search.
This approach runs both a keyword search and a semantic search, and then re-ranks the blended results.
The keyword search nails the exact terms, while the semantic search catches the meaning.
While it often gives the highest accuracy, it also doubles the infrastructure cost.
When deciding to implement this, consider whether your users value speed or accuracy, and whether your finance department wants to pay for that added accuracy.
Sometimes, the full context simply doesn't exist in a sub-point. Hierarchical search solves this and generally takes two forms:
Parent Document: You break your document into smaller chunks, but after you match a small chunk, you return the entire parent chunk. You get the precision of a small match with the context of the larger passage. Use this when matching sentences only make sense with more context. You should avoid this if your documents are already short.
RAPTOR: This stands for Recursive Abstractive Processing for Tree-Organized Retrieval. It is most helpful when your document is very nested, almost like a tree, which is common in legal documents. The process involves indexing and summarizing sections, then embedding and storing those summaries all the way to the top. By doing this, you convert multi-hop searches into a linear search, searching at different levels of hierarchy at once. However, be aware that you actually burn a lot of tokens doing this, and it will not be cheap.
Graph search is useful when your information doesn't exist in one single document, but rather in the relationship between documents.
A graph consists of nodes (things in the graph) and edges (the relationship between the things).
Previous strategies will never help you find out that one chunk is two degrees separated from another chunk.
If your query needs this relational information, you map your company's knowledge base into the graph during ingestion by inferring all the relationships.
It is incredibly interesting, but it requires substantial effort, and you should avoid it if your documents are simple and self-contained.
Which one should you use?
Start with hybrid search. It's the safest default for most enterprise content, because real document libraries rarely stick to one pattern of language. Reach for keyword search alone only when your users search in fixed codes or clause numbers. Reach for hierarchical or graph search only once you've confirmed the problem is genuinely about context or connections, not just retrieval accuracy. Adding a graph or a summary tree before you need one is a good way to spend a lot of engineering time solving a problem you don't have yet.
Not sure which of these five fits your documents?
Most teams don't get retrieval wrong because they picked the wrong strategy on paper. They get it wrong because their document library doesn't follow one single pattern: part of it needs exact-match precision, part of it needs meaning, and some of it only makes sense once you follow the connections between clauses. Working that mix out before you build is most of the job.
If you're scoping a RAG project and want the retrieval layer right from the start, or you're mid-build and match quality still isn't there, email us at hello@redairship.com. Tell us what your documents look like and where the matches are going wrong. We'll give you a straight answer on whether the fix is a different retrieval strategy, a storage problem, or something else entirely.
Rate this insight
Was this article useful?
Click to rate
