[Official FAQ] Share your experience or follow-up questions below.

Retrieval-Augmented Generation (RAG) means retrieving relevant passages from your own documents and giving them to the model along with the question, so answers are grounded in your data.

• 1. Load documents: PDFs, web pages, Markdown, and so on, converted to clean text. Scanned PDFs need OCR first.
• 2. Chunk: split text into passages, ideally by headings or paragraphs, with a small overlap. Too small loses context, too large dilutes relevance. Tune the size on your own data.
• 3. Embed: use an embedding model to turn each chunk into a vector.
• 4. Store: in a vector database such as Chroma, Qdrant, FAISS or pgvector.
• 5. Retrieve: embed the user's question and fetch the top matching chunks.
• 6. Generate: put the chunks in the prompt and instruct the model to answer only from them and to say when the answer is not there.

If you prefer less code, tools like Open WebUI knowledge bases, AnythingLLM, Dify, LlamaIndex or LangChain can handle much of this.

Tips for better results

• Keep source references and ask the model to cite them so you can verify.
• Add a reranker to refine the top results.
• Hybrid search (vector plus keyword) usually helps with names and codes.
• Use the same embedding model for indexing and querying.

When answers are poor, first check what was actually retrieved. Bad retrieval is more common than a bad language model.

What documents are you trying to search? Share your use case below.