RAG-based Q&A
Multi-stage retrieval for technical documentation under a 2,000-token budget; ranked 3rd of 141 teams with 0.978 correctness.
Till it's ready, here's the GitHub repo: github.com/simran-malik/ask ↗
The problem
Answer single-turn questions about the RapidFire AI platform from its documentation — but pass the model no more than 2,000 tokens per query, prompt included. Real document collections are far larger than any context window, so the constraint forces real retrieval engineering rather than stuffing everything into the prompt.
What I built
A multi-stage pipeline: dense retrieval with FAISS, then a cross-encoder reranker, then an LLM filter that keeps only the context that matters. I used grid searches over chunk sizes and generator models to tune it, and chain-of-thought prompts to improve completeness.
Results
- Rank 3 of 141 teams on the leaderboard.
- Correctness 0.978, faithfulness 1.0, completeness 0.902.
- Retrieval F1@5 up from 0.74 to 0.95; completeness up 16% from the prompting changes.