Autonomous agents
Google-CloudVertexBot is a bot operated by Google that browses the web on its own to finish multi-step tasks. Google-CloudVertexBot crawls sites on the site owners' request when building Vertex AI Agents. It honours robots.txt, and a request claiming to be Google-CloudVertexBot can be checked against a published IP range file.
Request says
"Google-CloudVertexBot"
Checked against
Google's published IP ranges
Live check
Paste the address from the log line. It is checked against what the operator publishes.
Reference
| User agent | Google-CloudVertexBotToken only: match on the substring. |
|---|---|
| Operator | |
| Purpose | Build and manage AI models for businesses employing Vertex AI |
| Respects robots.txt | Honourssource |
| Verification method | IP rangesPublished IP range file Crawls on behalf of a Vertex AI customer building their own grounded agent. Blocking it blocks that customer, not Google Search. https://developers.google.com/static/search/apis/ipranges/special-crawlers.json |
| Crawl pattern | Not published |
| Robots.txt tokens | Google-CloudVertexBotCloudVertexBot |
| Operator documentation | https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers |
Google-CloudVertexBot crawls sites on the site owners' request when building Vertex AI Agents.
robots.txt
Paste one at the root of your domain. Rules are per token.
User-agent: Google-CloudVertexBot
User-agent: CloudVertexBot
Disallow: /User-agent: Google-CloudVertexBot
User-agent: CloudVertexBot
Allow: /User-agent: Google-CloudVertexBot
User-agent: CloudVertexBot
Allow: /
Disallow: /account/
Disallow: /checkout/
Disallow: /searchEdit the Disallow paths to match your own URLs.
Blocking Google-CloudVertexBot stops automated sessions that fill forms, click through flows and read pages the way a logged-in user would. Rate limits and challenges usually work better here than a robots.txt line.
Build a file for every crawlerFAQ
Google-CloudVertexBot is a bot operated by Google that browses the web on its own to finish multi-step tasks. Google-CloudVertexBot crawls sites on the site owners' request when building Vertex AI Agents. It honours robots.txt, and a request claiming to be Google-CloudVertexBot can be checked against a published IP range file.
Yes. The operator documents that Google-CloudVertexBot reads robots.txt and follows Disallow rules for its own token. Rules take effect on the next fetch of the file, which can be hours or days after you publish them.
Crawls on behalf of a Vertex AI customer building their own grounded agent. Blocking it blocks that customer, not Google Search. The reference is published at https://developers.google.com/static/search/apis/ipranges/special-crawlers.json.
Add these user-agent tokens (Google-CloudVertexBot, CloudVertexBot) to robots.txt with Disallow: /. If the crawler ignores robots.txt, match the user agent at your CDN or reverse proxy and return 403 there instead.
Blocking Google-CloudVertexBot stops automated sessions that fill forms, click through flows and read pages the way a logged-in user would. Rate limits and challenges usually work better here than a robots.txt line.