AI search crawlers

PerplexityBot

PerplexityBot is a bot operated by Perplexity that indexes pages so an AI answer engine can cite them. Crawls sites to surface as results in Perplexity. It honours robots.txt, and a request claiming to be PerplexityBot can be checked against a published IP range file.

PerplexityBot at a glance

Reference data for the PerplexityBot crawler
User agentMozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot
OperatorPerplexity
PurposeSearch result generation.
Respects robots.txtYes, documentedsource
Verification methodPublished IP range file

Perplexity publishes the crawler ranges as JSON. Independent researchers have reported traffic that claims to be PerplexityBot from outside those ranges, so the IP check matters here more than most.

https://www.perplexity.com/perplexitybot.json
Crawl patternNot published
Robots.txt tokensPerplexityBot
Operator documentationhttps://docs.perplexity.ai/guides/bots

Crawls sites to surface as results in Perplexity.

robots.txt

Allow or block PerplexityBot

Paste one of these into the robots.txt file at the root of your domain. Rules are per token, so a block on one crawler leaves every other bot untouched.

Block PerplexityBot
User-agent: PerplexityBot
Disallow: /

Blocks every path for this crawler only.

Allow PerplexityBot
User-agent: PerplexityBot
Allow: /

Explicit allow. Useful when a wildcard rule above it would otherwise catch this crawler.

Allow, except the pages you never want quoted
User-agent: PerplexityBot
Allow: /
Disallow: /account/
Disallow: /checkout/
Disallow: /search

Edit the Disallow paths to match your own account, checkout and search URLs.

What blocking actually costs you

Blocking PerplexityBot takes the site out of the index behind its answers, so it stops being cited there. If referral traffic from AI answers matters to you, block the training crawler instead and leave this one alone.

Block every crawler in the directory at once
Verification

Is that really PerplexityBot?

A user-agent header is a string the client chooses. Scrapers copy PerplexityBot precisely because site owners allow it. Perplexity publishes the crawler ranges as JSON. Independent researchers have reported traffic that claims to be PerplexityBot from outside those ranges, so the IP check matters here more than most.

Paste the IP address from your access log below. You will see whether it belongs to a hosting provider, a residential proxy pool or a Tor exit, plus the ASN that announces it, which is what tells you whether the claim holds up.

Other crawlers run by Perplexity

Related crawlers

Agentscan

Identify crawlers automatically instead of by hand

Agentscan checks a request against published crawler ranges, reverse DNS and hosting data, then returns a verdict your edge can act on. One call per request, no range files to keep up to date.

FAQ

PerplexityBot questions

PerplexityBot is a bot operated by Perplexity that indexes pages so an AI answer engine can cite them. Crawls sites to surface as results in Perplexity. It honours robots.txt, and a request claiming to be PerplexityBot can be checked against a published IP range file.