KNOWLEDGE

In the KNOWLEDGE section, you define what your chatbot actually knows. This is where you add the knowledge base.

The KNOWLEDGE section contains five sub-sections:

• Documents — Upload files for your chatbot to learn from

• Web Sources — Scrape or crawl websites

• Text — Add custom text content directly

• Q&A — Create structured question-and-answer pairs

• WordPress — Import content from your WordPress site

---

Documents

What it is: Upload files containing information you want your chatbot to learn from.

Supported file formats:

• PDF files (.pdf)

• Text files (.txt)

• Markdown files (.md)

File size limit: Maximum 10MB per file

Storage characters count toward your plan limit (500K Free / 10M Basic / 20M Essential / 30M Business). Larger documents consume more of your storage budget — see Where to Find Your Usage.

How to use:

1. Go to KNOWLEDGE → Documents

2. Click "Add Data" → "Add Document"

3. Either drag and drop your file into the upload area, or click "Browse File" to select a file

4. Wait for the upload to complete (you'll see a progress bar)

5. The document will be processed automatically

6. Once processing is complete, the document will appear in your knowledge base

Best for:

• Product manuals and documentation

• Company policies and procedures

• Training materials

• Any structured text content you have in file format

Tips:

• Make sure your documents are well-formatted and readable

• Break large documents into smaller, focused files for better results

---

Web Sources

What it is: Extract content from web pages to train your chatbot. You can either scrape a single page or crawl multiple pages from a website.

2 options available:

Option A: Crawl Website

• What it does: Discovers and crawls all important pages from your website using a three-strategy approach.

• Discovery strategies (combined automatically):

  1. Sitemap discovery — checks robots.txt and common paths like /sitemap.xml for a published sitemap

  2. HTML crawling — follows internal links breadth-first up to a configurable depth

  3. Pattern analysis — recognizes common URL patterns (/blog/, /docs/, date-based, numeric-ID) to surface related pages the crawler might have missed

• How it works:

  1. Enter your website URL (e.g., https://example.com)

  2. The system analyzes your site and returns deduplicated, domain-constrained URLs

  3. Filter the list with include / exclude patterns if needed

  4. Select up to 10 pages at a time to train your chatbot

  5. Click Save to process the selected pages

Each successfully processed URL counts as 1 toward your per-chatbot URL limit (100 Free / 2,000 Basic / 5,000 Essential / 5,000 Business).

Option B: Scrape Single Page

• What it does: Extracts content from only the specific URL you provide

• How it works:

  1. Select "Scrape single page" option

  2. Enter the exact URL you want to scrape

  3. Click "Add"

  4. The page content is automatically extracted and processed

Best for:

• Your company website

• Blog posts and articles

• Product pages

• Help documentation pages

• Any publicly accessible web content

Tips:

• Make sure the URLs are publicly accessible (not behind login)

• For crawling, start with your homepage URL

• You can resync URLs later if content changes (available on Essential and Business plans)

Status states — every URL/document moves through these states (visible in the Documents list):

Status	Meaning
pending	Created, queued for processing
processing	Text being extracted, chunked, and indexed in the vector store
completed	Indexed and available for retrieval
failed	Processing error — hover the badge to see the message; you can retry

URL Resync (Essential & Business)

When the source page changes, click the Resync button on a URL document to fetch the latest content:

• Same chunk count → existing chunks updated in-place (IDs reused)

• Different chunk count → old chunks deleted, fresh ones uploaded

The page returns either "updated" or "replaced" on success. Free and Basic plans can re-add the URL manually instead.

---

Text

What it is: Manually add custom text content directly to your chatbot's knowledge base.

How to use:

1. Go to KNOWLEDGE → Text

2. Click "Add Data" → "Add Text"

3. Enter a title for your text content (e.g., "Company Overview", "Product Features")

4. Type or paste your text content in the text area

5. Click "Add" to save

6. The text will be processed and added to your knowledge base

Best for:

• Custom information not available in documents or on websites

• Company-specific details

• Product descriptions

• Frequently asked questions

• Any text content you want to add directly

Tips:

• Use descriptive titles to organize your content

• Break long text into multiple entries for better organization

• Keep text focused and relevant to what users might ask

---

Q&A

What it is: Create structured question-and-answer pairs that your chatbot will use to respond to specific questions.

How to use:

1. Go to KNOWLEDGE → Q&A

2. Click "Add Data" → "Questions with Answers"

3. Enter a title for your Q&A set (e.g., "Product FAQs", "Support Questions")

4. Add question-and-answer pairs:

   • Question: The question users might ask

   • Answer: The exact answer you want your chatbot to provide

5. Click "Add" to save

6. The Q&A pairs will be processed and added to your knowledge base

Best for:

• Frequently asked questions (FAQs)

• Common support questions

• Product-specific questions

• Any questions where you want exact, controlled answers

Tips:

• Write questions as users would naturally ask them

• Provide clear, complete answers

• Review and update Q&As regularly as your business evolves

---

WordPress

What it is: Import content directly from your WordPress website, including posts, pages, and custom post types.

How to use:

1. Go to KNOWLEDGE → WordPress

2. Click "Add Data" → "WordPress Content"

3. If you have WordPress sites connected, select the site you want to import from

4. Choose the post type (Posts, Pages, or custom post types)

5. Browse and select the specific posts or pages you want to import

6. Click "Add Content" to process

7. The selected WordPress content will be imported and added to your knowledge base

Best for:

• WordPress website owners

• Blog posts and articles

• Any content hosted on WordPress

Requirements:

• Your WordPress site must be connected to AI Botkit (via the AI Botkit WordPress plugin)

WordPress Content Auto-Sync (Essential & Business)

On Essential and Business plans, AI Bot Kit listens to your WordPress site for content changes. Whenever you publish, update, or delete a post or page that's already in the chatbot's knowledge base, the WordPress plugin webhooks AI Bot Kit at /api/wordpress/sync-content. The chatbot:

1. Authenticates the call via the site's authToken

2. Finds the matching document by postId + siteId

3. Re-fetches the post HTML, re-chunks it, and updates the index — or removes it on delete

You don't have to manually re-import after every edit. The change is reflected in the chatbot's answers within seconds of WordPress finishing the save.