> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dubot.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge

> Ingest, scope, search, and read product documentation

## Build one reviewable workspace corpus

Dubot can ingest product guidance from a public Help Center or supported document uploads, then
make a defined scope available to wizards, Guidance, and the Resource Center.

<CardGroup cols={3}>
  <Card title="Crawl a Help Center" icon="globe">
    Add a public source and collect the pages available under its configured path.
  </Card>

  <Card title="Upload documents" icon="file-text">
    Add PDF, DOCX, Markdown, and text files.
  </Card>

  <Card title="Scope retrieval" icon="search">
    Let an experience use all knowledge, selected sources, selected articles, or none.
  </Card>
</CardGroup>

<img src="https://mintcdn.com/dubot/xmKfaoq5qOSd2RjO/assets/product/knowledge-crawl-progress.png?fit=max&auto=format&n=xmKfaoq5qOSd2RjO&q=85&s=d88d95738c29bef4aaa8e8cd18a59e00" alt="A current Dubot knowledge-source crawl in progress" width="1440" height="900" data-path="assets/product/knowledge-crawl-progress.png" />

<Note>
  This demonstrates the live heartbeat, moving discovery and crawl counts, failures, and the
  existing corpus remaining available during a re-crawl.
</Note>

## Current product behavior

* Inspect sources and extracted articles for the workspace.
* Re-crawl an individual web source or sync all crawl sources.
* Enable weekly synchronization for crawl sources.
* Review article count, corpus size, sync state, and content no longer linked from its source.
* Remove a source and its extracted content.
* Search the configured scope during a wizard, Guidance, or Resource Center request.
* Read a matching article in bounded windows rather than loading the entire corpus into every
  request.

<Info>
  Runtime knowledge retrieval is implemented in the current product build. The previous public
  documentation that described retrieval as future work is no longer accurate.
</Info>

## Retrieval flow

```text theme={null}
public Help Center or uploaded documents
  -> deterministic extraction
    -> workspace corpus
      -> experience-specific scope
        -> search matching articles
          -> read the relevant article content
```

The current runtime exposes server-side knowledge search and article reading to eligible
assistants. It does not imply unrestricted access to every article, authenticated Help Center
crawling, or a particular future retrieval architecture.

## Current boundaries

| Area                   | Current behavior                                                                   |
| ---------------------- | ---------------------------------------------------------------------------------- |
| Web access             | Publicly accessible Help Center pages                                              |
| Crawl rules            | Source robots directives are honored                                               |
| Uploads                | PDF, DOCX, Markdown, and TXT                                                       |
| Freshness              | Manual sync or optional weekly sync for web sources                                |
| Private sources        | Authenticated crawling is not part of the current contract                         |
| Scope                  | All content, selected sources, selected articles, or none                          |
| Resource Center search | Published article cards only; raw search text is not retained as analytics content |

<Warning>
  Ingestion and access are separate decisions. Review source ownership, exclusions, sensitivity,
  freshness, and experience-level scope before publishing knowledge-backed behavior.
</Warning>