Can I allow some AI crawlers and block others?

Yes. Agentscan identifies AI crawlers by name, including GPTBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot, and supports a customer-managed allowlist on top of the global one, so you can permit the ones you want and block the rest.

Does it detect headless browsers and Playwright or Puppeteer?

Yes. It combines User-Agent tells such as HeadlessChrome, Playwright, Puppeteer and Selenium with client-side signals such as webdriver and missing browser surfaces, and weighs them against the IP origin. Headless from a datacenter range is the canonical malicious automation.

What is JA4 and why does it matter?

JA4 is a fingerprint of the TLS client hello. Because it reflects how the client actually speaks TLS, it is much harder to fake than a User-Agent header, so a bot pretending to be Chrome can still be told apart by its TLS fingerprint.

Will it slow down my site?

No. Verdicts are computed from signals collected at the edge and cached in Redis keyed by IP, User-Agent and JA4, so repeat lookups return in under 50ms. A real human in a real browser is never blocked on origin alone, so a person on a VPN is still classified human.

IPScanner

Products

IP DetectionVPN, proxy, Tor and hosting checks on any IP AgentscanTell humans, bots and AI agents apart ProvenanceLocation checks with a tamper-evident audit log Email securityCatch throwaway, mistyped and dead signup emails

Integrations

Cloudflare Worker WordPress plugin WooCommerce Traefik plugin Caddy module nginx, Apache, HAProxy Form gate Signup Gate MCP server All integrations

Detection servicesVPN Detection Proxy Detection Geolocation ASN Data WHOIS Info

Bot and AI crawler control

AI agent and bot detection, keep scrapers out, let Googlebot in.

Agentscan sorts every visitor into people, verified crawlers, AI agents and bot scripts, then applies your rule to each. Install it as a WordPress plugin, a Cloudflare Worker or at your proxy.

Start free See where it installs

Verified crawlers always passNo card

Human12

Known bot4

AI agent3

Malicious automation5

Human12

Known bot4

AI agent3

Malicious automation5

Speed

Decided before your visitor blinks.

Every check answers in 50 to 100 ms, so your pages and signups never wait on it.

IPScanner checkmedian, at your edge: ~85 ms
A blinkhuman eye: 300 ms
A fast page loadGoogle's “good” mark: 2.5 s

How it works

Install, watch, then decide

01
Install where traffic arrives
The WordPress plugin, the Cloudflare Worker, or a plugin for Traefik, Caddy or nginx. No changes to your app.
02
See who visits
Your dashboard splits traffic into people, verified crawlers, AI agents and scripts, with the networks they come from.
03
Set a rule per class
Allow, flag or block. Blocking runs on credits, and your free credits cover it to start.

Use cases

Where Agentscan earns its keep

Content and SEO protection
Allow the answer engines you want indexed and block training scrapers, without nuking Googlebot.
Anti-scraping and scalping
Stop headless automation hammering pricing, inventory and ticket endpoints.
Account security
Add friction to credential-stuffing and signup abuse driven by automation from masked IPs.
AI-traffic policy
Decide per crawler whether GPTBot, ClaudeBot, Perplexity and the rest may read your pages.

Integrations

No code on the common setups

Pick where your site runs. Each install reports to the same dashboard.

All integrations

Every request, sorted

Know which visitors are people

Humans, verified crawlers, AI agents and scripts, told apart by the network, the browser and what each operator publishes.

Create an account Browse the crawler directory

Safari 18 · iPhoneHumanAllow

GooglebotVerified botAllow

GPTBotAI agentFlag

HeadlessChrome/141Malicious automationBlock

ClaudeBotAI agentFlag

Chrome 141 · WindowsHumanAllow

python-requests/2.32Malicious automationBlock

BingbotVerified botAllow

Safari 18 · iPhoneHumanAllow

GooglebotVerified botAllow

GPTBotAI agentFlag

HeadlessChrome/141Malicious automationBlock

ClaudeBotAI agentFlag

Chrome 141 · WindowsHumanAllow

python-requests/2.32Malicious automationBlock

BingbotVerified botAllow

GPTBot · AI agentFlag

Verdicts

Four classes, one action each

Every request lands in exactly one class with a confidence and a recommended action.

Human
A real person in a real browser, so the request is allowed.
Known Bot
A verified good bot such as Googlebot or Bingbot that is allowed and never accidentally blocked.
AI Agent
An identified AI fetcher such as GPTBot or ClaudeBot that you flag and decide on per policy.
Malicious Automation
Headless automation with no good identity, often from a masked origin, so the request is blocked.

Under the hood

Identity from the network up to the request

Agentscan fuses the shared IP engine with request-level fingerprints and a verified allowlist.

IP origin class
Starts from the engine verdict of datacenter, VPN, proxy or clean residential, because masked origin changes everything.
Headless and automation tells
Detects HeadlessChrome, Playwright, Puppeteer, Selenium and scripted clients from the User-Agent plus client-side signals like webdriver.
JA4 TLS fingerprint
A TLS client fingerprint that survives a spoofed User-Agent, separating real browsers from impostors.
Header consistency
Scores how browser-like the header set is, since real browsers send Accept, Accept-Language and Accept-Encoding together.
Verified allowlist
Googlebot, Bingbot and friends are confirmed by forward-confirmed reverse DNS rather than by a spoofable User-Agent string.
AI-crawler awareness
Knows GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more, so you can allow or block them by name.

For your developers

One call for your developers

Call it from the edge with signals collected by the snippet. Verdicts are Redis-cached for low latency.

POST/v1/agentscan/check

curl -X POST 'https://ipscanner.io/v1/agentscan/check' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"ip":"198.51.100.7","user_agent":"Mozilla/5.0 HeadlessChrome/120","ja4":"t13d1516h2_8daaf6152771_...","headless_flags":{"webdriver":true},"headers":{"Accept":"*/*"}}'

response

{
  "class": "malicious_automation",
  "confidence": 0.9,
  "action": "block",
  "signals": { "network_origin": "datacenter", "headless": true }
}

Your 25,000 free credits cover 2,500 checks. Call it from your server: the key does not belong in a browser.

View API docs OpenAPI spec

SDKs for Node.js, Python, Go, PHP, Ruby and Rust

FAQ

Questions, answered

Agentscan does forward-confirmed reverse DNS. It looks up the IP's PTR record, checks it ends in an official domain such as googlebot.com, then resolves that hostname back and confirms it matches the IP. A spoofed Googlebot User-Agent from the wrong network fails this and is not treated as a known bot.

See which visitors are people

Install it, watch your traffic sorted by class, then choose what to block.

Start free See pricing

No credit card to start

AI agent and bot detection, keep scrapers out, let Googlebot in.

Decided before your visitor blinks.

Install, watch, then decide

Install where traffic arrives

See who visits

Set a rule per class

Where Agentscan earns its keep

Content and SEO protection

Anti-scraping and scalping

Account security

AI-traffic policy

No code on the common setups

Know which visitors are people

Four classes, one action each

Identity from the network up to the request

IP origin class

Headless and automation tells

JA4 TLS fingerprint

Header consistency

Verified allowlist

AI-crawler awareness

One call for your developers

Questions, answered

See which visitors are people