AI search crawlers
PerplexityBot is a bot operated by Perplexity that indexes pages so an AI answer engine can cite them. Crawls sites to surface as results in Perplexity. It honours robots.txt, and a request claiming to be PerplexityBot can be checked against a published IP range file.
Request says
"PerplexityBot/1.0"
Checked against
Perplexity's published IP ranges
Live check
Paste the address from the log line. It is checked against what the operator publishes.
Reference
| User agent | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot |
|---|---|
| Operator | Perplexity |
| Purpose | Search result generation. |
| Respects robots.txt | Honourssource |
| Verification method | IP rangesPublished IP range file Perplexity publishes the crawler ranges as JSON. Independent researchers have reported traffic that claims to be PerplexityBot from outside those ranges, so the IP check matters here more than most. |
| Crawl pattern | Not published |
| Robots.txt tokens | PerplexityBot |
| Operator documentation | https://docs.perplexity.ai/guides/bots |
Crawls sites to surface as results in Perplexity.
robots.txt
Paste one at the root of your domain. Rules are per token.
User-agent: PerplexityBot
Disallow: /User-agent: PerplexityBot
Allow: /User-agent: PerplexityBot
Allow: /
Disallow: /account/
Disallow: /checkout/
Disallow: /searchEdit the Disallow paths to match your own URLs.
Blocking PerplexityBot takes the site out of the index behind its answers, so it stops being cited there. If referral traffic from AI answers matters to you, block the training crawler instead and leave this one alone.
Build a file for every crawlerFAQ
PerplexityBot is a bot operated by Perplexity that indexes pages so an AI answer engine can cite them. Crawls sites to surface as results in Perplexity. It honours robots.txt, and a request claiming to be PerplexityBot can be checked against a published IP range file.
Yes. The operator documents that PerplexityBot reads robots.txt and follows Disallow rules for its own token. Rules take effect on the next fetch of the file, which can be hours or days after you publish them.
Perplexity publishes the crawler ranges as JSON. Independent researchers have reported traffic that claims to be PerplexityBot from outside those ranges, so the IP check matters here more than most. The reference is published at https://www.perplexity.com/perplexitybot.json.
Add the PerplexityBot token to robots.txt with Disallow: /. If the crawler ignores robots.txt, match the user agent at your CDN or reverse proxy and return 403 there instead.
Blocking PerplexityBot takes the site out of the index behind its answers, so it stops being cited there. If referral traffic from AI answers matters to you, block the training crawler instead and leave this one alone.