AskAiml
Sign in Get started

Advanced RAG: Self-RAG, CRAG, HyDE, GraphRAG, multi-query

When and why to reach for query rewriting, self-critique, graph-based retrieval, and multi-index fanout.

📚 Module 2 7 min read Not started

Why this matters

Naive RAG fails in well-known ways: irrelevant retrieval, hallucination on out-of-corpus questions, and inability to multi-hop reason. The 5 patterns in this lesson are how senior engineers fix those failures. They also distinguish your portfolio from the 95% of candidates who only built "embed-search-answer" pipelines.

Learning objectives

  1. Implement HyDE to improve queries before search.
  2. Implement multi-query expansion.
  3. Implement Self-RAG (decide whether to retrieve, score chunks, regenerate if poor).
  4. Implement Corrective RAG (CRAG) with web-search fallback.
  5. Build a tiny GraphRAG for multi-hop questions.

1. HyDE — hypothetical document embeddings

User queries are short. Documents are long. Their embeddings live in different "regions" of the space, hurting recall.

Trick: ask the LLM to write a hypothetical answer to the question. Embed the hypothetical answer (which looks like a document!) and use that as the query vector.

python
HYDE = """Write a 2-3 sentence factual answer to the question, even if you have to imagine it. This is for a search task."""

def hyde_search(question: str, k: int = 10):
    hypo = client.responses.create(
        model="gpt-4.1-mini",
        instructions=HYDE,
        input=question,
    ).output_text
    q_vec = embed(hypo)
    return vector_store.query(query_embeddings=[q_vec], n_results=k)

Cost: one extra cheap LLM call per query. Lift: typically 5-15% recall on factual QA. Free win.

When not to use HyDE: queries that are exact-match by nature (IDs, codes, names). Combine with BM25 to keep both.

2. Multi-query expansion

A user asks one question. You quietly turn it into 3 reformulations and search each.

python
EXPAND = """Rewrite the user's question into 3 alternative search queries that might find the same answer. Return as JSON array."""

def multi_query(q: str) -> list[str]:
    r = client.responses.parse(
        model="gpt-4.1-mini",
        instructions=EXPAND,
        input=q,
        text_format=list[str],
    )
    return [q] + r.output_parsed[:3]

def search_all(q: str, k: int = 10):
    queries = multi_query(q)
    all_hits: list[list[str]] = [vector_store.query(query_texts=[qq], n_results=k)["ids"][0] for qq in queries]
    return rrf(all_hits, k=60)

LangChain has MultiQueryRetriever if you do not want to write your own.

3. Self-RAG — let the agent decide

A naive retriever runs unconditionally on every query. Self-RAG (Asai et al., 2023) lets the model:

  1. Decide if retrieval is even needed (Retrieve?).
  2. Score retrieved chunks for relevance (IsRel?).
  3. After answering, judge if the answer is supported (IsSup?) and useful (IsUse?).
  4. If poor, regenerate with different chunks or no retrieval.

You implement this as a LangGraph state machine (you will learn LangGraph in Phase 3, but the concept is here):

[user_query]
    ↓
[decide_retrieve] ──no──→ [direct_answer] → END
    │ yes
    ↓
[retrieve_top_k]
    ↓
[score_chunks] ──low avg score──→ [reformulate_query] → loop
    │ ok
    ↓
[generate_answer]
    ↓
[judge_supported & useful] ──no──→ [regenerate] → END
    │ yes
    ↓
END

Pseudocode:

python
def self_rag(question: str) -> str:
    if not needs_retrieval(question):
        return direct_answer(question)
    chunks = retrieve(question)
    scores = [score_relevance(c, question) for c in chunks]
    if max(scores) < 0.5:
        question = reformulate(question)
        chunks = retrieve(question)
    answer = generate(question, chunks)
    if not judge_supported(answer, chunks):
        answer = regenerate(question, chunks)
    return answer

needs_retrieval, score_relevance, judge_supported are all 1-paragraph LLM-as-judge calls. Fast, cheap, much more reliable than naive RAG.

What if the local KB is poor? CRAG adds a fallback: if retrieved confidence is below a threshold, do a web search instead.

python
def crag(question: str) -> str:
    chunks = retrieve(question)
    score = avg_score(chunks)

    if score >= 0.7:
        return generate(question, chunks)
    elif score >= 0.4:
        # ambiguous: combine local + web
        web = web_search(question)
        return generate(question, chunks + web)
    else:
        # local KB cannot help
        web = web_search(question)
        return generate(question, web)

Web search options:

  • Tavily — built for LLMs, structured snippets. Free tier 1k/month.
  • Brave Search API — generous free tier, good quality.
  • SerpAPI / Serper — Google results.
  • Exa (formerly Metaphor) — semantic web search.
  • Perplexity API — answers + sources.
python
# uv add tavily-python
from tavily import TavilyClient
tv = TavilyClient(api_key=os.environ["TAVILY_API_KEY"])
res = tv.search("recent agentic AI papers", max_results=5, search_depth="advanced")

CRAG is essential for any agent that promises "current information."

5. GraphRAG — for multi-hop questions

Vector RAG fails on questions like "Which engineers work for the company that acquired Acme last year?" because the answer is not in any single chunk — it requires traversing relationships.

GraphRAG (Microsoft, 2024) builds a knowledge graph of entities and relations from your documents, then answers via graph traversal + community summaries.

The full Microsoft graphrag library is heavy. Here is a minimal version with NetworkX that demonstrates the idea:

python
# uv add networkx
import networkx as nx
import json

EXTRACT = """Extract entities and relations from the text as JSON:
{"entities":[{"id":"E1","type":"Person","name":"..."}],
 "relations":[{"src":"E1","tgt":"E2","label":"works_at"}]}"""

class Graph:
    def __init__(self): self.g = nx.MultiDiGraph()

    def ingest(self, text: str):
        out = client.responses.parse(
            model="gpt-4.1-mini",
            instructions=EXTRACT,
            input=text,
            text_format=dict,
        ).output_parsed
        for e in out["entities"]:
            self.g.add_node(e["id"], **e)
        for r in out["relations"]:
            self.g.add_edge(r["src"], r["tgt"], label=r["label"])

    def hop(self, start: str, max_hops: int = 2):
        return nx.ego_graph(self.g, start, radius=max_hops)

Then to answer:

  1. Identify entities mentioned in the question.
  2. Pull a small ego subgraph from each.
  3. Serialise the subgraph as text/JSON.
  4. Pass to LLM as context.

For real production, use:

  • Microsoft GraphRAG (full Python, includes community summaries).
  • LightRAG (HKU, 2024 — simpler, faster).
  • Neo4j + LLM Graph Builder (managed graph DB + ingest).
  • Memgraph with Cypher queries.

GraphRAG is heavy compute; reserve it for genuinely multi-hop corpora (legal, biomed, knowledge bases of organisations).

6. Combining advanced patterns — a 2026 reference architecture

[user query]
    ↓
[multi_query expand] ── 3 queries
    ↓
[hybrid retrieve] ── BM25 + dense for each → RRF
    ↓
[contextual chunks already had context prepended]
    ↓
[rerank top-50 with Cohere rerank-3]
    ↓
[Self-RAG relevance score] ── if low → CRAG → web search fallback
    ↓
[answer with citations]
    ↓
[judge supported] ── if no → regenerate
    ↓
END

Implement once. You can defend it for an hour in any interview.

Hands-on lab (5 hours)

Take your hybrid RAG from Lesson 2.3 and upgrade:

  1. Add HyDE as a flag on the retriever.
  2. Add multi-query expansion (3 reformulations).
  3. Wrap retrieval in a Self-RAG decision: should we retrieve, are chunks relevant, is answer supported.
  4. Add a CRAG path that calls Tavily when retrieval scores are low.
  5. Bonus: build a tiny GraphRAG index over a 50-page corpus and answer one multi-hop question.

Eval requirement: build a 30-question test set including:

  • 10 standard factual (vector wins).
  • 5 exact-ID (BM25 wins).
  • 5 multi-hop (GraphRAG wins).
  • 5 out-of-corpus (CRAG → web wins).
  • 5 unanswerable (model should refuse).

Compare your advanced pipeline against the naive pipeline on this set. Print a markdown table. This is portfolio gold.

Common pitfalls

  1. HyDE on ID queries. It hallucinates and ranks worse. Detect numeric/ID patterns and skip HyDE.
  2. Multi-query without dedup. RRF dedup happens automatically, but watch the 3x token cost.
  3. Self-RAG infinite loops. Cap regenerations at 2. Track in state["regen_count"].
  4. CRAG on private corpora. You leak the user's private question to a web search. Add a check.
  5. GraphRAG bloat. Extract only top-N entities per chunk; do not extract every noun.

Self-check

  1. Why does HyDE help on most queries but hurt on ID lookups?
  2. Where does Self-RAG decide to skip retrieval entirely?
  3. What is the failure mode CRAG specifically fixes?
  4. When does GraphRAG beat vector RAG and when does it lose?
  5. How would you measure that each of these techniques is contributing to the lift?

References

Sign in to save your progress and earn badges.