AI search crawlers
Amzn-SearchBot is a bot operated by Amazon that indexes pages so an AI answer engine can cite them. Amzn-SearchBot crawls pages for Amazon shopping and assistant search surfaces. Whether it honours robots.txt is undocumented, and a request claiming to be Amzn-SearchBot can be checked against a forward-confirmed reverse DNS lookup.
Request says
"Amzn-SearchBot"
Checked against
Amazon's reverse DNS
Live check
Paste the address from the log line. It is checked against what the operator publishes.
Reference
| User agent | Amzn-SearchBotToken only: match on the substring. |
|---|---|
| Operator | Amazon |
| Purpose | AI search crawlers |
| Respects robots.txt | No policy |
| Verification method | Reverse DNSReverse DNS, forward confirmed Part of the Amazon crawler family documented alongside Amazonbot. Verify with the same reverse-then-forward DNS check. |
| Crawl pattern | Not published |
| Robots.txt tokens | Amzn-SearchBot |
| Operator documentation | None published |
Amzn-SearchBot crawls pages for Amazon shopping and assistant search surfaces. It is documented alongside Amazonbot and verified the same way.
robots.txt
Paste one at the root of your domain. Rules are per token.
User-agent: Amzn-SearchBot
Disallow: /User-agent: Amzn-SearchBot
Allow: /User-agent: Amzn-SearchBot
Allow: /
Disallow: /account/
Disallow: /checkout/
Disallow: /searchEdit the Disallow paths to match your own URLs.
Blocking Amzn-SearchBot takes the site out of the index behind its answers, so it stops being cited there. If referral traffic from AI answers matters to you, block the training crawler instead and leave this one alone.
Build a file for every crawlerFAQ
Amzn-SearchBot is a bot operated by Amazon that indexes pages so an AI answer engine can cite them. Amzn-SearchBot crawls pages for Amazon shopping and assistant search surfaces. Whether it honours robots.txt is undocumented, and a request claiming to be Amzn-SearchBot can be checked against a forward-confirmed reverse DNS lookup.
Not documented. No operator statement says whether Amzn-SearchBot reads robots.txt, so add the rule but verify in your logs that the requests actually stop.
Part of the Amazon crawler family documented alongside Amazonbot. Verify with the same reverse-then-forward DNS check. The reference is published at https://developer.amazon.com/amazonbot.
Add the Amzn-SearchBot token to robots.txt with Disallow: /. If the crawler ignores robots.txt, match the user agent at your CDN or reverse proxy and return 403 there instead.
Blocking Amzn-SearchBot takes the site out of the index behind its answers, so it stops being cited there. If referral traffic from AI answers matters to you, block the training crawler instead and leave this one alone.