How Accurate Are 0xDoktor's Answers? Confidence Scoring and Human Fallback

How accurate is 0xDoktor?

Accuracy depends on two factors: the quality of your knowledge base and the clarity of user questions.

When a user's question maps clearly to content in your knowledge base, 0xDoktor is highly accurate — because it is not generating content, it is retrieving it. The language model's job is to compose a clear, coherent answer from retrieved chunks, not to invent information.

When a question is ambiguous, outside the scope of your KB, or requires recent information you have not ingested, 0xDoktor performs worse — but it handles these cases explicitly rather than fabricating an answer.

What is confidence scoring?

Every answer 0xDoktor generates comes with a confidence score — a numeric value between 0 and 1 that reflects how well the retrieved content matched the question.

A high confidence score (above 0.8) means the semantic search found strongly relevant chunks and the model had clear material to work with.

A low confidence score (below 0.5) means the retrieval step found weak matches. In this case, 0xDoktor may:

• Answer with a caveat that the information may be incomplete
• Decline to answer and suggest contacting a human
• Trigger automatic escalation to your support team (if copilot mode is enabled)

You configure the confidence threshold that triggers escalation in the admin dashboard.

What is the wasResolved flag?

After receiving an answer, users can indicate whether it resolved their question. This binary feedback — resolved or not resolved — is stored as the wasResolved flag on each conversation.

This data is valuable in two ways:

1. Quality feedback: a high "not resolved" rate on a particular topic signals a gap in your knowledge base
2. Continuous improvement: the admin dashboard surfaces questions with low resolution rates so you can add or update content

Over time, tracking wasResolved across your full conversation history gives you a clear picture of where your documentation is strong and where it needs work.

What is copilot mode?

Copilot mode is a human escalation system built into 0xDoktor. When a conversation is flagged — either by the confidence score dropping below threshold or by the user explicitly requesting a human — copilot mode is activated.

In copilot mode:

• The conversation is surfaced to a human agent in the admin dashboard
• The agent can see the full conversation context
• The agent can reply directly through the dashboard
• The handoff is transparent to the user

Copilot mode is optional. If you do not have a human support team available, you can configure 0xDoktor to instead show a contact form or email address when escalation is needed.

What about hallucinations?

0xDoktor's architecture is designed to minimize hallucinations by grounding every answer in retrieved content. The model cannot add information that is not present in the retrieved chunks without it being apparent.

That said, no AI system is completely hallucination-free. 0xDoktor mitigates this risk by:

• Requiring source chunks for every answer
• Exposing confidence scores so uncertain answers are visible
• Providing wasResolved tracking to catch patterns of incorrect answers
• Supporting human review of flagged conversations

If you notice a pattern of incorrect answers on a specific topic, the solution is usually to review and improve the relevant documentation in your knowledge base.

How do I improve accuracy over time?

The most effective way to improve accuracy is to treat your knowledge base as a living document:

1. Monitor the admin dashboard for low-confidence questions
2. Review conversations marked as "not resolved" by users
3. Add or update documents to cover the gaps
4. Re-ingest the updated content

0xDoktor's reporting tools make this feedback loop visible. Most customers see significant accuracy improvements within the first month of active knowledge base maintenance.