RAG-powered KB chatbot — "Ask Our Docs"

After Phase 2 you have all the pieces. This project assembles them into a polished, defensible RAG product.

What you ship

A chatbot for a real document corpus (your company handbook, a course's PDF set, a legal codebook, or the FastAPI docs). Web UI (Streamlit or Next.js), citations on every answer, evaluation dashboard, and "I do not know" path for out-of-corpus questions.

Tech stack

• Ingest: Unstructured (or LlamaParse for tabular PDFs).
• Chunking: structure-aware Markdown splitter + parent-child + contextual chunking (Anthropic 2024).
• Embeddings: text-embedding-3-small. Optionally voyage-3-large for v2.
• Stores: ChromaDB (dev) and pgvector (production parallel build).
• Hybrid: bm25s + dense + RRF.
• Rerank: Cohere rerank-3 (free tier 1k/month).
• LLM: gpt-4.1-mini for synthesis; claude-haiku-4-5 for context-prepend (with prompt caching).
• Evaluator: RAGAS v0.2 (Faithfulness, Context Recall, Context Precision, Answer Relevancy, Factual Correctness).
• Frontend: Streamlit (fastest path) or Next.js + FastAPI for fancier.
• Tracing: LangSmith (free tier).

Architecture (mermaid)

flowchart LR
  U[User] --> UI[Streamlit Chat]
  UI -->|/ask| API[FastAPI]
  API --> Q[Multi-query expand + HyDE]
  Q --> R1[BM25 search]
  Q --> R2[Vector search (Chroma + pgvector)]
  R1 --> RRF[RRF Fuse]
  R2 --> RRF
  RRF --> RR[Cohere rerank-3]
  RR --> GEN[LLM synthesis with citations]
  GEN --> UI
  GEN --> TR[LangSmith]
  EV[RAGAS eval suite] --> CI[GitHub Actions]

Step-by-step

Day 1 — Ingest

• Pick the corpus. Convert all to Markdown via Unstructured.
• Save raw + cleaned text to data/raw/ and data/clean/.
• Tag every doc with source, tenant, updated_at, tags.

Day 2 — Chunking

• Implement parent-child split. Parents = 2000-token chapters. Children = 300-token sections.
• Add contextual chunking that prepends a 1-sentence "where this chunk fits" using Claude Haiku with prompt caching. Save costs to a CSV.
• Index children in Chroma + pgvector with metadata {parent_id, source, h1, h2, h3}.

Day 3 — Hybrid retrieval

• Build a bm25s index over the same children.
• Implement rrf(...). Confirm fused recall beats either alone on 30 hand-written queries.

Day 4 — Rerank + synthesis

• Add Cohere rerank-3 on top-50 candidates.
• LLM synthesis prompt: "Answer using ONLY the provided sources. Cite as [n]. If not in sources, say 'I do not know.'"
• Make sure parent chunks are passed (not children) for context.

Day 5 — Evaluation

• Generate 50 Q/A with RAGAS TestsetGenerator. Hand-review.
• Run evaluate(...) with all 6 metrics. Store CSV per run.
• Add a baseline (vector-only) and the full pipeline. Tabulate Δ.

Day 6 — UI + tracing

• Streamlit page: input box, streaming answer, expandable sources panel.
• LangSmith env vars set. Streamlit calls a FastAPI /ask that wraps the chain.

Day 7 — Polish

• Write the post: problem → architecture diagram → eval results → cost breakdown → lessons learned.
• Record a 60-second demo video.
• Push to GitHub with a make command that runs the eval gate.

Acceptance checklist

• 100% citations for every claim.
• Faithfulness ≥ 0.85, Context Recall ≥ 0.80 on golden set.
• p95 latency under 1.5s on a laptop.
• "I do not know" path triggers cleanly on 5 out-of-corpus questions.
• LangSmith trace link in README.
• Mermaid diagram + UMAP plot of embeddings + RAGAS table in README.
• One-liner Docker docker compose up runs the whole stack.

What hiring managers are looking for

• A clear before/after RAGAS table proving each technique helped.
• Cost transparency: how many tokens per query? Caching savings?
• Honest "what I would do next" section.

This single project demonstrates 80% of "RAG Engineer" interview rubrics. Polish it.