How to Set Up RAG for 0xDoktor: Preparing Your Knowledge Base for Accurate Answers

What this guide covers

Retrieval-Augmented Generation (RAG) quality depends heavily on what you put in — and how you structure it. This guide covers:

• What content to ingest and what to leave out
• How to structure documents for better chunk quality
• How to verify retrieval is working as expected
• How to iterate on your knowledge base based on confidence data

Step 1: Decide what to ingest

Not all documentation makes good RAG content. Good candidates:

• FAQ pages: Tightly scoped question-answer pairs produce excellent retrieval results.
• Feature documentation: Specific, factual descriptions of how your product works.
• Step-by-step procedures: Numbered steps with clear prerequisites.
• Policy documents: Definitive statements about what is and isn't permitted.

Poor candidates:

• Marketing copy: Vague, promotional language does not match how users phrase support questions.
• Changelog entries: Versioned lists of changes are rarely what users are searching for.
• Raw database exports: Unstructured tabular data without context confuses retrieval.
• Overly long documents: Documents over 10,000 words should be split into logical sections before ingestion.

Step 2: Structure your documents for chunking

0xDoktor chunks documents during ingestion. How you structure your documents affects chunk quality.

Use clear headings: Each heading creates a natural chunk boundary. Answers that span multiple sections should be consolidated under a single heading when possible.

Write self-contained sections: Each section should make sense read in isolation. Avoid writing "as described above" — the chunk containing "above" may not be retrieved alongside the current chunk.

Keep answers close to questions: If you have a FAQ format, keep the question and its complete answer in the same section. Do not split them across headings.

Example of good structure:

## How do I reset my password?

To reset your password:
1. Go to Settings → Security.
2. Click "Reset password".
3. Enter your current email address.
4. Check your inbox for the reset link (valid for 24 hours).

If you do not receive the email within 5 minutes, check your spam folder.

Example of poor structure:

## Password Management

Passwords can be reset using the procedure described in the Security section.
See also: Account Settings.

Step 3: Ingest in priority order

Start with the 10–20 documents that answer the most common user questions. Check your support inbox or help desk to identify the top questions — these are your highest-value ingestion targets.

After the initial corpus is ingested, run test queries from the admin dashboard to confirm that common questions return high-confidence answers.

Step 4: Check chunk coverage

After ingestion, navigate to your knowledge base in the admin dashboard. Each document shows its chunk count. A document with 0 chunks failed to ingest (usually a parsing error). A document with an unexpectedly high chunk count may be too long and should be split.

Run a sample of user questions through the test interface. For each question, examine:

• Confidence score: below 0.5 → the topic may not be covered, or the document structure is poor
• Sources: check that the retrieved sources are the right documents for the question
• Answer accuracy: read the answer critically — does it correctly answer the question?

Step 5: Iterate based on analytics

After going live, use the conversation analytics in the admin dashboard to identify low-confidence patterns:

1. Export conversations with confidence < 0.5.
2. Group them by topic.
3. For each topic cluster, add or improve the relevant documentation.
4. Re-ingest the updated documents.

Most knowledge bases see a significant confidence improvement after the first iteration cycle (typically 2–3 weeks after launch when real user data is available).

Common pitfalls

Over-ingesting: More is not always better. Ingesting loosely related content adds noise to retrieval, which can reduce confidence for specific queries.

Not updating after product changes: Stale documentation is worse than no documentation. Set a reminder to audit your knowledge base whenever your product changes significantly.

Ignoring the sources field: Always check which sources are being retrieved for your most important queries. If the wrong source is retrieved, the answer will be wrong regardless of the LLM quality.