WhatsApp RAG

End-to-end local RAG system for WhatsApp group chat exports. Parses raw logs to JSONL, builds a Chroma vector index with local embeddings, optionally reranks with a local cross-encoder, and answers with a local LLM. Nothing leaves the machine.

Architecture

• Data: WhatsApp .txt → normalized JSONL (one message per line)
• Indexing: JSONL → conversational chunks → embeddings → Chroma persistent store (citations in metadata)
• Retrieval: embed query → retrieve top-k → optional cross-encoder rerank
• Generation: grounded prompt → local LLM answers concisely with inline citations

Tech Stack

• Python 3.10/3.11 with CUDA 12.1/12.2
• SentenceTransformers BAAI/bge-large-en-v1.5
• ChromaDB persistent vector store
• llama-cpp-python with Llama 3.1 8B GGUF
• Optional: cross-encoder/ms-marco-MiniLM-L-6-v2 for reranking
• Gradio web UI