mirror of
https://github.com/Shubhamsaboo/awesome-llm-apps.git
synced 2026-09-14 14:29:41 +08:00
Thejesh
a855f2cae2
- corrective_rag: route .pdf URLs to PyPDFLoader (WebBaseLoader was running an HTML parser over the default arXiv PDF and embedding decoded garbage); and only ingest when the source changes — Streamlit re-runs the script on every question, so it was deleting the collection and re-embedding the whole document each time. - agentic_rag_math_agent: KB gate was 'similarity > 0.', which passes for almost any query (cosine sim of OpenAI embeddings is ~always >0), so an arbitrary KB answer was served as authoritative with 'do not recalculate'. Require >= 0.80. - knowledge_graph_rag_citations: semantic_search did Neo4j CONTAINS on the whole question, which never matches an entity; tokenize and match any term case-insensitively. - ai_blog_search: create the Qdrant collection before constructing QdrantVectorStore (which validates and 404s on a fresh instance). All four compile-check clean. Co-authored-by: thejesh23 <thejesh23@users.noreply.github.com>
🧠 Math Tutor Agent – Agentic RAG with Feedback Loop
This project implements an Agentic-RAG architecture to simulate a math professor that solves JEE-level math questions with step-by-step explanations. The system smartly routes queries between a vector database and web search, applies input/output guardrails, and incorporates human feedback for continuous learning.
📌 Features
- ✅ Input Guardrails (DSPy): Accepts only academic math questions.
- 📚 Knowledge Base Search: Uses Qdrant Vector DB with OpenAI Embeddings to match known questions.
- 🌐 Web Fallback: Integrates Tavily API when no good match is found.
- ✍️ GPT-4.1 Explanations: Generates step-by-step math solutions.
- 🛡️ Output Guardrails: Filters for correctness and safety.
- 👍 Human-in-the-Loop Feedback: Users rate answers (Yes/No), logged for future learning.
- 📊 Benchmarking: Evaluated on JEEBench dataset with adjustable question limits.
- 💻 Streamlit UI: Interactive dashboard with multiple tabs.
🚀 Architecture Flow
📚 Knowledge Base
- Dataset: JEEBench (HuggingFace)
- Vector DB: Qdrant (with OpenAI Embeddings)
- Storage: Built with
llama-indexto persist embeddings and perform top-1 similarity search
🌐 Web Search
- Uses Tavily API for fallback search when the KB doesn't contain a good match
- Fetched content is piped into GPT-4o for clean explanation
🔐 Guardrails
- Input Guardrail (DSPy): Accepts only math-related academic questions
- Output Guardrail (DSPy): Blocks hallucinated or off-topic content
👨🏫 Human-in-the-Loop Feedback
- Streamlit UI allows students to give 👍 / 👎 after seeing the answer
- Feedback is logged to a local JSON file for future improvement
📊 Benchmarking
- Evaluated on 50 random JEEBench Math Questions
- Current Accuracy: 66%
- Benchmark results saved to:
benchmark/results.csv
🚀 Demo
To run the app with Streamlit:
streamlit run app/streamlit.py