# Spoofed AI Crawler - Definition and How to Tell a Real Bot

> A spoofed AI crawler claims to be GPTBot or ClaudeBot but is not. How to check a bot with vendor IP ranges and Web Bot Auth, and what the result means.

Source: https://angeo.dev/spoofed-ai-crawler-definition/
Updated: 2026-10-17

Canonical definition

# Spoofed AI Crawler - Definition and How to Tell a Real Bot

**A spoofed AI crawler is a request that says "I am GPTBot" (or ClaudeBot, or PerplexityBot) in its User-Agent header, but does not come from that vendor.** The header is one line of text that anyone can send.

**In one sentence**
The User-Agent is a claim; the vendor's published IP range or a valid signature is the proof.

## Why it matters for a store

Spoofed bots cause three problems:

- **Wrong numbers.** If you count every "GPTBot" request, your AI traffic report includes scrapers that only borrowed the name.
- **Wrong decisions.** A WAF rule that trusts the User-Agent lets fake bots through, while a rule that blocks by name can also block the real one.
- **Wasted load.** Aggressive scrapers often hide behind well-known bot names because many sites allow them.

## How to check a bot

There are two kinds of proof. A store can use whichever the vendor publishes.

| Check | How it works | Strength |

| **IP range** | The vendor publishes the addresses its crawler uses. The request's source IP must be in that list. | Good, if your server sees the real client IP |
| **Signed request (Web Bot Auth)** | The bot signs each request (RFC 9421). The site checks the signature against the vendor's published key. | Strongest, but few vendors sign today |
| **User-Agent only** | Read the header | No proof at all |

## What vendors publish (October 2026)

| Vendor | IP ranges | Signed requests |

| OpenAI | Yes | Yes (ChatGPT agent) |
| Anthropic | Yes  -  one shared list for ClaudeBot, Claude-User and Claude-SearchBot | No |
| Perplexity | Yes, per bot | No |
| Google | Yes, for user-triggered fetchers such as Google-Agent. Google-Extended is a token, not a crawler. | No |
| Apple, Meta, ByteDance, Mistral, DuckDuckGo | No AI crawler list | No |

A match in a shared list proves the vendor, not which of its bots sent the request. Say that plainly in any report.

## Four results, not two

An honest check has four answers:

- **verified**  -  the IP is in the vendor's range, or the signature is valid;
- **failed / spoofed**  -  it is not;
- **unknown**  -  no decision: the list could not be loaded, or the server only sees a private proxy address;
- **unsupported**  -  the vendor publishes no way to check.

"Unknown" is not "spoofed". Behind a CDN or load balancer, set up trusted proxy headers first, or every check ends as unknown.

## A Magento example

[AI Agent Activity](https://angeo.dev/modules/agent-activity/) downloads the IP lists of OpenAI, Anthropic, Perplexity and Google-Agent once a day and marks each AI crawler request as verified or spoofed. It uses the IP for the check and never stores it. [Robots.txt AEO](https://angeo.dev/modules/robots-txt-aeo/) adds two CLI checks: `angeo:robots:verify-bot-ip` for an address and `angeo:robots:verify-bot-request` for a signed request. Both modules report. Blocking belongs in your WAF or CDN.

## Questions

Can I trust the User-Agent header?
No. It is a claim that any client can send. Use the vendor's published IP ranges or a request signature as proof.
Should I block spoofed AI crawlers?
Usually yes, at the WAF or CDN, once you are sure the check sees the real client IP. Do not block real search bots by mistake: they bring AI citations.
Why does my check say unknown so often?
Most often the server only sees a proxy or load balancer address. Configure trusted proxy headers so the application sees the client IP.
Does blocking the IPs of a real AI crawler opt me out?
It stops the crawler, but it also stops it reading robots.txt, which is where your preference lives. Use robots.txt rules to opt out.

## Related

- [GPTBot vs OAI-SearchBot](https://angeo.dev/gptbot-vs-oai-searchbot/)

- [Web Bot Auth - definition](https://angeo.dev/web-bot-auth-definition/)

- [AI crawlers reference](https://angeo.dev/docs/ai-crawlers/)

- [Robots.txt AEO module](https://angeo.dev/modules/robots-txt-aeo/)

- [AI Agent Activity module](https://angeo.dev/modules/agent-activity/)

- [Glossary](https://angeo.dev/glossary/)

## Sources

- [OpenAI: Overview of OpenAI crawlers](https://platform.openai.com/docs/bots)

- [Anthropic: crawler IP ranges (bots.json)](https://claude.com/crawling/bots.json)

- [Perplexity: Perplexity crawlers](https://docs.perplexity.ai/guides/bots)

- [angeo/module-robots-txt-aeo README](https://github.com/angeo-dev/module-robots-txt-aeo)

Verified 16 October 2026. Vendors change their IP feeds and signing support; check the sources before you rely on the table. Disclosure: we publish the open-source modules linked above.