About ListurboBot
You may have noticed ListurboBot in your WAF logs. This page explains who we are, what we do, and how to allow us through your firewall.
1Who we are
Listurbo is a Generative Engine Optimization (GEO) SaaS platform. We help brands get cited, recommended, and mentioned in AI engines like ChatGPT, Gemini, Claude, Perplexity, and Grok.
We do not build a training corpus. We do not redistribute fetched content to third parties. ListurboBot only retrieves content when one of our customers explicitly asks us to evaluate a site they own or are auditing.
2When ListurboBot fetches your site
ListurboBot fetches a page only when a Listurbo user actively runs one of these tools against your URL:
- Sitemap discovery — reads /robots.txt and /sitemap.xml to enumerate the site’s public pages.
- AI crawler rule check — reads your robots.txt to see which AI crawlers it allows and which it blocks. The crawlers we look for are GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, Googlebot, Google-Extended, Bingbot, PerplexityBot, Perplexity-User, xAI-Bot.
- Schema markup audit — parses HTML for JSON-LD (Q&A, step-by-step guides, articles and company details) so we can suggest improvements.
- Content freshness scan — compares page last-modified + content hash against the previous evaluation snapshot to detect stale pages.
Fetch volume is bounded by the customer’s plan quota. We respect Crawl-Delay if your robots.txt declares one, and we never run aggressive concurrent crawls.
3Our user-agent string
Every request ListurboBot makes carries this exact User-Agent header. Allowlist by this string and you’ll catch all of our traffic and nothing else.
ListurboBot/1.0 (+https://listurbo.com/help/bot)Identifies us — bot name we self-declare. Use this token alone for substring matching.
Crawler version. Industry-standard format. Will increment as we evolve our fetch behavior.
Link to this exact page. The + prefix is the standard convention so log parsers can extract the URL.
4Allow us in robots.txt (one line)
Add the snippet below to your site’s /robots.txt file. ListurboBot reads robots.txt on every fetch session and respects this directive.
User-agent: ListurboBot Allow: /
Tip: If you already have a global User-agent: * with Allow: /, ListurboBot is already permitted at the robots.txt layer — but your WAF may still block us. See section 5.
5WAF allowlist steps (3 platforms)
If your robots.txt already allows ListurboBot but we’re still blocked, your WAF (Web Application Firewall) is rejecting us at the network layer. Pick your WAF platform and follow the steps below.
6Why this matters for your GEO
Blocking ListurboBot also means you are blocking GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, Googlebot, Google-Extended, Bingbot, PerplexityBot, Perplexity-User, xAI-Bot in most cases — the same WAF rules that target our user-agent string typically reject other AI crawlers as well. When AI engines cannot read your site’s content, your brand will not be cited, recommended, or mentioned in their answers. Opening your site to all legitimate AI crawlers is the core action of GEO optimization.
7Contact us
If you have questions about ListurboBot’s behavior, want to report unusual traffic, or need help adjusting your WAF, reach out via the channels below.
