AI search crawlers

Bingbot

Bingbot is a bot operated by Microsoft that indexes pages so an AI answer engine can cite them. Bingbot indexes pages for Bing Search. It honours robots.txt, and a request claiming to be Bingbot can be checked against a published IP range file or a forward-confirmed reverse DNS lookup.

Bingbot at a glance

Reference data for the Bingbot crawler
User agentMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
OperatorMicrosoft
PurposeSearch indexing that also feeds Copilot answers
Respects robots.txtYes, documentedsource
Verification methodPublished IP ranges or reverse DNS

Reverse DNS must resolve to a search.msn.com host and forward-resolve back to the same IP. Microsoft also publishes the ranges as JSON.

https://www.bing.com/toolbox/bingbot.json
Crawl patternContinuous, controllable via Bing Webmaster Tools crawl control.
Robots.txt tokensBingbot
Operator documentationhttps://www.bing.com/webmasters/help/how-to-verify-bingbot-3905dc26

Bingbot indexes pages for Bing Search. The same index grounds Microsoft Copilot answers, so a Bingbot block removes a site from Copilot citations as well as from Bing results.

robots.txt

Allow or block Bingbot

Paste one of these into the robots.txt file at the root of your domain. Rules are per token, so a block on one crawler leaves every other bot untouched.

Block Bingbot
User-agent: Bingbot
Disallow: /

Blocks every path for this crawler only.

Allow Bingbot
User-agent: Bingbot
Allow: /

Explicit allow. Useful when a wildcard rule above it would otherwise catch this crawler.

Allow, except the pages you never want quoted
User-agent: Bingbot
Allow: /
Disallow: /account/
Disallow: /checkout/
Disallow: /search

Edit the Disallow paths to match your own account, checkout and search URLs.

What blocking actually costs you

Blocking Bingbot takes the site out of the index behind its answers, so it stops being cited there. If referral traffic from AI answers matters to you, block the training crawler instead and leave this one alone.

Block every crawler in the directory at once
Verification

Is that really Bingbot?

A user-agent header is a string the client chooses. Scrapers copy Bingbot precisely because site owners allow it. Reverse DNS must resolve to a search.msn.com host and forward-resolve back to the same IP. Microsoft also publishes the ranges as JSON.

Paste the IP address from your access log below. You will see whether it belongs to a hosting provider, a residential proxy pool or a Tor exit, plus the ASN that announces it, which is what tells you whether the claim holds up.

Other crawlers run by Microsoft

Related crawlers

Agentscan

Identify crawlers automatically instead of by hand

Agentscan checks a request against published crawler ranges, reverse DNS and hosting data, then returns a verdict your edge can act on. One call per request, no range files to keep up to date.

FAQ

Bingbot questions

Bingbot is a bot operated by Microsoft that indexes pages so an AI answer engine can cite them. Bingbot indexes pages for Bing Search. It honours robots.txt, and a request claiming to be Bingbot can be checked against a published IP range file or a forward-confirmed reverse DNS lookup.