Author and self-reviewer: Michael Fisher, ChattyBox maintainer. Published and checked August 4, 2026. This is a practical risk-reduction guide, not an independent review or a claim of perfect answer accuracy.

1. Define the support failures you are trying to prevent​

Treat hallucination as an operational failure, not only a model problem. In support, it includes an answer that invents a fact, combines two true facts into a false instruction, cites an irrelevant page, or applies a public rule to a customer-specific case.

For example:

• A visitor asks, “Can I get a refund after 45 days?” The bot confidently says yes because it retrieved an old promotion rather than the current refund policy.
• A customer asks why their invoice was declined. The bot infers a payment reason from generic billing documentation even though it cannot see the account.
• A user asks for a configuration step. The answer links to a source citation, but the cited page describes a different product version.

Make a short risk register before configuring prompts. List your high-impact topics—payments, privacy, security, eligibility, data deletion, legal commitments, and account-specific issues—then decide which topics the bot may answer, must qualify, or must hand off. The NIST Generative AI Profile is a useful primary-source reference for treating confabulation as a risk to manage across the system lifecycle.

2. Build a small, trusted source set before broadening coverage​

Retrieval-augmented generation (RAG) gives a support bot relevant context instead of relying solely on a language model's general knowledge. It helps, but it cannot repair inaccurate, ambiguous, or outdated material. Start with sources a support owner has approved:

1. Current help-center articles, product documentation, policies, and status pages.
2. Canonical pages for each policy; exclude duplicated campaign pages, old release notes, drafts, and staging URLs.
3. Pages with an owner and review date for high-risk claims.
4. Clear, task-oriented articles that state prerequisites, limits, exceptions, and the date or version they apply to.

Do not index private account pages, internal notes, checkout flows, or content the bot is not authorized to disclose. If two pages say different things, adding both does not create a reliable answer—it creates an unresolved decision for retrieval to guess at.

For a website implementation overview, see how RAG works for a website chatbot. When preparing a crawl, use the content scraping guide to select a sitemap or an intentional list of URLs, then compare the completed index against your approved source inventory.

3. Improve retrieval quality before tuning the wording​

Most support hallucinations that look like “bad answers” begin with missing or weak context. Diagnose the retrieval result separately from the final response.

For every important test question, record the expected source, the retrieved URLs or passages when the product exposes them, and the answer. If the expected source is absent, improve the source set or the source page before adjusting the bot's tone. Useful fixes include:

• Split a long page that combines unrelated policies; give each policy a direct heading and a canonical URL.
• Put the answer, conditions, and exceptions in the same section rather than scattered across navigation or linked PDFs.
• Use product names, plan names, and version identifiers consistently in the question set and documentation.
• Re-scrape after edits, redirects, migrations, and releases; a correct live page is not proof that the index contains its current text.
• Test paraphrases, misspellings, short questions, and multi-part questions—not just the wording used in the docs.

The OWASP Top 10 for LLM and generative AI applications also identifies vector and embedding weaknesses as a system risk. In practice, that means treating retrieved text as untrusted input: review what it says, where it came from, and whether it should be eligible to guide a support answer.

4. Require claim-level support and check citations​

Configure and evaluate the bot to answer from retrieved, approved context; to avoid filling gaps from general knowledge; and to attach a source link when it makes a factual claim. Then verify more than the presence of a citation.

Ask three questions for each answer:

1. Entailment: Does the cited passage actually support every material claim in the answer?
2. Applicability: Is it the right product, plan, region, audience, and version for this customer?
3. Freshness: Is the source still current, and does a newer canonical page supersede it?

Citations reduce risk, but they do not prove correctness. A link can be broken, stale, irrelevant, or only partially supportive. Citation checking must therefore include opening the source, reading the cited section, and comparing it with the answer—not merely asserting that a source chip appeared. For implementation patterns and UX considerations, see AI chatbots with source citations.

5. Make abstention and fallback a successful outcome​

A safe support bot needs a useful response for questions it cannot support. Set a clear fallback for no evidence, conflicting evidence, low-confidence retrieval, private account data, or high-impact advice.

Instead of: “Your annual plan can be cancelled any time with a full refund,” use: “I can’t confirm the refund terms for your plan from the available help content. Please review the current refund policy or contact support so we can check your account.”

The fallback should:

• Say what it cannot verify without inventing a reason.
• Link to the relevant canonical help page when one exists.
• Offer a human-support route and preserve the customer's question.
• Avoid asking customers to share passwords, payment-card data, authentication codes, or other sensitive information in chat.

This is not a poor experience. It prevents a confident but costly answer and gives the support team evidence of a documentation gap. The AI customer support chatbot guide explains how to pair public answers with an escalation path for account-specific and sensitive requests.

6. Resolve conflicting and stale sources deliberately​

Create a source-of-truth rule for each changing topic. For example, the current policy page may override blog posts; the status page may override a troubleshooting article during an incident; and a versioned document may apply only to customers on that version.

When a conflict appears, do not ask the model to reconcile it. Remove or exclude the retired source, update redirects and canonicals, publish the corrected policy in one owned location, and re-scrape. Record the change date and test the old question again. If the business cannot determine which rule applies, the bot should abstain and escalate rather than selecting the answer that sounds most likely.

7. Escalate decisions, not just unanswered questions​

Some questions are answerable from public docs but still inappropriate for autonomous support. Route these to a person or a controlled account workflow: disputes, cancellations with exceptions, security incidents, data requests, regulated advice, contract interpretation, identity verification, and actions that change money, access, or data.

Give agents the conversation, cited sources, product context, and a reason code such as no_source, source_conflict, account_specific, or high_impact. That makes handoffs faster and lets you distinguish a retrieval failure from a policy that should never have been automated.

8. Run a reproducible pre-launch test plan​

Do not launch because a few friendly questions worked. Freeze a versioned evaluation sheet and run it in an incognito browser against the deployed environment after every material source, prompt, model, or integration change.

Use at least these cases:

For each row, save the date, environment, source-index version, question, retrieved URLs, response, cited URLs, pass/fail result, and reviewer. Mark a response as failed when it makes an unsupported material claim—even if it sounds helpful. Check the launch checklist for the corresponding crawl, browser, key, and deployment checks.

9. Review incidents like product defects​

When a customer reports a wrong answer, preserve the evidence available before changing anything: the exact question, answer, citations, retrieved URLs or passages when exposed, source versions, configuration version, customer impact, and escalation outcome. Then classify the cause:

• Coverage gap: the correct source was never indexed.
• Retrieval miss: the source existed but was not retrieved or was outranked.
• Grounding failure: the source was retrieved but the answer went beyond it.
• Citation failure: the answer's link did not support the claim.
• Content governance failure: sources conflicted or were stale.
• Routing failure: the bot should have escalated.

Assign an owner and a corrective action—edit or retire content, change source scope, add a regression test, strengthen an escalation rule, or improve the review workflow. Re-run the original prompt and related paraphrases before closing the incident. Trends in these classifications are more useful than a single “accuracy” number because they point to the layer that needs work.

10. Reusable support-bot safety template​

Copy this into your launch ticket or operating runbook and complete the bracketed fields:

Support topic: [for example, cancellations]
Business owner / last reviewed: [name, YYYY-MM-DD]
Canonical source URLs: [URLs]
Excluded or superseded URLs: [URLs]
Allowed answer scope: [facts the bot may state]
Must-escalate conditions: [account-specific, exceptions, high-impact cases]
Fallback message: [plain-language abstention and support route]

Evaluation cases
- Supported question + expected source: [...]
- Paraphrase / typo: [...]
- Missing-source question: [...]
- Stale or conflicting-source question: [...]
- Account-specific or high-impact question: [...]
- Prompt-injection attempt: [...]

For every result, record: question | retrieved URLs | answer | cited URLs |
entailment check | freshness check | pass/fail | reviewer | configuration/index version

Incident owner: [name]
Review cadence: [weekly at launch, then monthly]

Use the template alongside the scraping workflow and launch checklist. Reducing hallucinations is a continuing support-quality practice: curate the knowledge, verify the evidence, abstain when it is insufficient, and learn from every failure.

Sources​

• NIST AI Risk Management Framework: Generative AI Profile (NIST AI 600-1)
• NIST AI Risk Management Framework
• OWASP Top 10 for LLM and Generative AI Applications

Author and technical reviewer: Michael Fisher, ChattyBox maintainer. Published and technically checked July 10, 2026. The repeatable checks below are an implementation tutorial, not a performance or accuracy benchmark.

1. Add the Root theme component​

Create src/theme/Root.tsx in your Docusaurus site:

import React, { useEffect, type ReactNode } from 'react';

const WIDGET_ID = 'chattybox-widget';

export default function Root({ children }: { children: ReactNode }) {
  useEffect(() => {
    if (document.getElementById(WIDGET_ID)) return;

    const script = document.createElement('script');
    script.id = WIDGET_ID;
    script.src = 'https://chattybox.ai/widget.js';
    script.async = true;
    script.dataset.apiKey = 'YOUR_API_KEY';
    script.dataset.apiUrl = 'YOUR_CHAT_API_URL';
    script.dataset.chattyboxWidget = 'true';
    document.body.appendChild(script);
  }, []);

  return <>{children}</>;
}

The stable chattybox-widget ID is the important part. React Strict Mode can remount effects during development, and Docusaurus changes routes without replacing the document. The guard makes both cases idempotent.

Use the API URL shown by your ChattyBox project rather than copying an example deployment. See the widget installation reference for the current attributes and the Docusaurus product guide for source-selection and evaluation guidance.

2. Keep the loader mounted​

Do not put this script inside an individual docs page or layout that Docusaurus replaces during navigation. The swizzled Root component wraps the application for its lifetime, so the widget remains available as visitors move between guides and references.

If your site already has src/theme/Root.tsx, merge the effect into the existing component rather than replacing authentication, analytics, or other providers.

3. Account for Content Security Policy​

A restrictive policy needs to allow:

• https://chattybox.ai in script-src for the widget loader.
• Your configured chat API origin in connect-src.
• https://fonts.googleapis.com in style-src and https://fonts.gstatic.com in font-src if the widget font is not already available.
• Inline component styles in style-src for the current widget build.

Start from your existing policy and add only the origins you actually use. Do not replace a restrictive policy with a broad wildcard.

4. Reproduce the integration checks​

In your Docusaurus project, add the Root wrapper above and run:

bun install
bun run start

Then verify:

1. Open two different docs routes without a full browser refresh.
2. Run document.querySelectorAll('#chattybox-widget').length after each navigation. It must remain 1.
3. Ask a question answered by an indexed page and confirm the response links to that page.
4. Ask an unsupported question and confirm the assistant falls back instead of inventing a source.
5. Test the launcher at a narrow mobile viewport and check that it does not cover navigation or pagination controls.

The script-count check proves duplicate prevention. It does not prove retrieval quality. Use a representative question set and the scraping guide to validate source coverage before launch.

What to monitor after launch​

Record unresolved questions, incorrect citations, stale source pages, and routes where the launcher obscures site controls. Re-test after Docusaurus theme upgrades because changes to navigation and content layout can affect placement even when the loader remains correct.

For a broader rollout sequence, use the documentation chatbot implementation checklist and launch checklist.

Sources​

• ChattyBox widget installation reference
• Docusaurus swizzling documentation
• MDN Content Security Policy guide

Author and technical reviewer: Michael Fisher, ChattyBox maintainer. Published and technically checked July 10, 2026. The verification protocol below is an implementation tutorial; no traffic, deflection, or answer-accuracy result is claimed.

1. Choose the correct override directory​

For Material for MkDocs, point custom_dir at an overrides directory:

site_name: My documentation
theme:
  name: material
  custom_dir: overrides
  features:
    - navigation.instant

For the built-in theme, use a separate custom theme directory:

site_name: My documentation
theme:
  name: mkdocs
  custom_dir: custom_theme

Keeping the variants separate makes it clear which upstream base template is being extended.

2. Extend the scripts block​

Create overrides/main.html for Material or custom_theme/main.html for the built-in theme:

{% extends "base.html" %}

{% block scripts %}
  {{ super() }}
  <script
    id="chattybox-widget"
    src="https://chattybox.ai/widget.js"
    async
    data-api-key="YOUR_API_KEY"
    data-api-url="YOUR_CHAT_API_URL"
    data-chattybox-widget="true"
  ></script>
{% endblock %}

{{ super() }} preserves scripts provided by the theme, including navigation and search behavior. Omitting it can make the widget appear to work while silently breaking the documentation interface. The stable ID also gives you a direct duplicate-loader assertion.

Use project-specific values from the widget installation reference. The MkDocs AI chatbot guide covers source scope, crawl behavior, and the questions to evaluate before publishing.

3. Build both variants​

From your MkDocs project:

python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt

cd material
mkdocs build --strict

cd ../vanilla
mkdocs build --strict

--strict turns warnings into failures, which catches broken navigation and configuration issues before the widget is evaluated.

4. Run the browser checks​

Serve each variant and verify:

1. Built-in search still opens and returns the expected documentation result.
2. Material instant navigation changes routes without duplicating the widget loader.
3. document.querySelectorAll('#chattybox-widget').length remains 1 after several route changes.
4. At a 390 px viewport, the launcher does not cover search, navigation, or next/previous controls.
5. A supported question cites the expected page, and an unsupported question produces a conservative fallback.

The first four checks validate integration behavior. The fifth depends on the pages you index and requires a representative evaluation set; follow the scraping guide and launch checklist rather than treating a successful script load as proof of answer quality.

5. Preserve a restrictive CSP​

Allow the widget host in script-src, the configured API origin in connect-src, and the font origins in style-src and font-src when needed. The current widget injects component styles, so an otherwise strict policy also needs an explicit inline-style strategy. Avoid wildcard source lists.

Maintenance checklist​

Rebuild both theme variants after MkDocs or Material upgrades, because upstream template block names can change. Keep the script ID stable, preserve super(), and re-run the search, instant-navigation, mobile-overlap, and source-citation checks.

For end-to-end rollout planning, use the documentation chatbot implementation checklist.

Sources​

• ChattyBox widget installation reference
• MkDocs theme customization
• Material for MkDocs customization
• MDN Content Security Policy guide

The product is built for teams that want a practical AI support layer without migrating documentation or rebuilding their site stack. Add your website URL, let ChattyBox crawl published content, test real questions, then install the widget with a script tag.

Why We Built It​

Most website chatbots either require a heavy sales-led setup or answer too broadly from model training data. ChattyBox focuses on the content you already publish, so visitors can ask natural-language questions and still verify the answer against your documentation.

That makes it useful for documentation sites, SaaS help centers, developer docs, CMS pages, and product websites where trust matters.

What Comes Next​

This blog will share practical guides on using AI to reduce support tickets, building reliable RAG chatbots for websites, and adding AI assistants to documentation without losing source control.

Start with the documentation AI chatbot guide or read the installation docs to add ChattyBox to your site.