You can expand your Agent’s knowledge base in multiple ways:
- Connect site content via a sitemap
- Upload documents (PDF)
You can create a knowledge base by opening an Agent and selecting Knowledge base. From there, click Connect website to start adding content to the knowledge
base.
Add a single URL
Use this if you only want to add information from a single web page.
Web scraping
If the website doesn’t have a sitemap, scraping is possible, but not recommended.
- More expensive
- Less clean data
- Less reliable results
Only use this as a last resort.
Sitemap indexing (recommended)
Add a sitemap (often found at /sitemap.xml, /sitemap, or /sitemap_index.xml).
This is the most reliable and efficient method.
- Cleaner data
- Cheaper indexing
- Keeps knowledge updated automatically
Automatic sitemap reanalysis
- With the Agent Workspace you can fetch updates from your sitemap every 24 hours if you enable automatic reanalysis.
- Make sure your sitemap includes last-updated timestamps. Without it,
automatic indexing can result in unexpected costs.
- ⚠️ Incorrect or incomplete sitemaps can lead to unnecessary costs. Always
validate your sitemap.
Adding documents
You also have the option to upload PDFs or add plain text within the same overview.
