Crawl your knowledge base into GoHighLevel's AI agents domain-wide
Domain-wide crawl scope reads the whole help center
Overview
GoHighLevel’s web crawler offers three scopes: an exact URL, a path, or the whole domain. Domain-wide is the one you want for a knowledge base, since the other two leave articles behind.
Their enhanced crawler handles tabs, accordions, and lazy-loaded sections, which matters more than it sounds: a real help center puts plenty of content behind exactly those, and simpler crawlers miss it.
What you can do
- Domain-wide crawl scope reads the whole help center
- Copes with tabs, accordions, and lazy-loaded sections
- Your articles stay in HelpDocs, with nothing to export
- Feeds the same content to every AI agent you run
Setting it up
- In GoHighLevel, go to AI Agents, then Knowledge Base, then Web Crawler
- Paste your knowledge base URL: https://your-slug.helpdocs.io
- Set the scope to domain-wide rather than exact URL or path-based
- Run the crawl and check the page count looks right
Requirements
- Your knowledge base must be public, so the crawler can read it
More AI Tools integrations
- HubSpot Service Hub Enables HubSpot to access your documentation by automatically adjusting your robots.txt file settings
- Jotform AI Agents Train a Jotform AI agent on your knowledge base, ten pages at a time
- Kaily.ai Add your knowledge base to Kaily.ai as a sitemap data source, synced daily
- Kustomer Add your knowledge base to Kustomer's AI agent as a public URL source
Start building your documentation infrastructure.
Create, organize, translate, and connect your docs without the admin sprawl of a support suite.
Start free 14-day trial
Migrate fromNo credit card required.
