Only 18% of Magento Product Pages Are Readable by AI. Here’s the Data

We ran an automated AEO (AI Engine Optimization) readiness scan across 462 live Magento storefronts. The headline: most are hard for AI systems to read — the very systems that increasingly sit between a shopper and a purchase, from ChatGPT shopping and Perplexity to Google’s AI Mode. Here is what the data shows, why it matters, and what a store owner can do about it.

What AEO is, and why it’s not just SEO

Traditional SEO primarily optimises for ranking in a list of search results. AEO optimises for being selected, cited, or acted upon by an AI system. The technical signals overlap, but they are not identical. Robots.txt access rules, structured data, and machine-readable site metadata increasingly determine whether an AI system can confidently retrieve, trust, and recommend a product — and whether an autonomous shopping agent can act on it. This study measures exactly those signals across the live Magento landscape.

What we scanned, and how

We built the scan specifically for Magento storefronts, using our own open-source audit tooling, and ran it read-only against public pages. It targeted 462 domains. Of those, 458 were reachable and 446 were confirmed to be running Magento — that confirmed-Magento set is the denominator for every store-level figure below. During the run, 13 stores returned rate-limiting responses (429/503); those pages were treated conservatively rather than being counted as failures.

For each store, the scanner:

• fetched and parsed robots.txt, checking each AI crawler against explicit and wildcard rules;
• confirmed the store was running Magento;
• checked for an llms.txt file at the site root;
• located one product page where possible;
• extracted any JSON-LD and Microdata structured data on that page;
• validated the required Product fields, including offers.availability;
• recorded the result against ten weighted AEO signals.

No store was modified; this was a read-only, outside-in assessment of exactly what a crawler would encounter.

Finding 1 — 12% of Magento stores block AI crawlers outright

Access is the first gate. If a store’s robots.txt disallows an AI crawler — explicitly or via a wildcard — nothing downstream matters, because the agent never reads the page.

• GPTBot blocked: 52 of 446 stores (12%)
• ClaudeBot blocked: 54 of 446 stores (12%)
• PerplexityBot blocked: 9 of 446 stores (2%)

To put that in human terms: roughly one Magento store in eight is shutting the two largest AI crawlers out of its catalog entirely. The pattern is telling. GPTBot and ClaudeBot are blocked at very similar rates, which strongly suggests these are not deliberate editorial decisions but the side effect of broad “block all AI scrapers” rules copied from a template or a security plugin. PerplexityBot slips through far more often — most likely because it is simply absent from those same blocklists, not because anyone chose to welcome it. In other words, a store’s AI visibility is frequently being decided by a config file nobody revisited, not by strategy.

Finding 2 — 87% of Magento stores have no llms.txt

The llms.txt convention gives an AI system a clean, curated map of a site: what the business is, which pages matter, and how the content should be understood. For many stores, publishing one is among the simplest AEO improvements to implement — and adoption is still rare enough that doing it is a genuine differentiator.

389 of 446 stores (87%) have no llms.txt at all. Put plainly: nearly nine out of ten Magento stores give AI systems no explicit guidance about which pages represent the business. Only around one store in eight currently exposes this signal — so publishing one puts you ahead of most Magento storefronts we analysed. For a Magento merchant this is close to free: the file can be generated from catalog and CMS data and kept current automatically.

Finding 3 — 82% of product pages ship no JSON-LD Product schema

To assess product markup fairly, we narrowed to the 223 stores where a product page could be positively identified. Everything in this section uses that 223 as its denominator. The primary split is simple and mutually exclusive: a product page either carries JSON-LD Product markup or it does not.

• No JSON-LD Product markup: 182 of 223 (82%) — of which 33 fall back on Microdata and 149 expose no product schema at all
• Has JSON-LD Product: 41 of 223 (18%)
• Of those 41, missing offers.availability: 10 (24%)

This is the most consequential finding. The vast majority of product pages ship no JSON-LD Product markup — the format that is widely supported by modern search engines and AI systems. Within that 82%, a smaller group (33 stores, 15% of the 223) still exposes Microdata: technically machine-readable, but a weaker and less consistently interpreted signal. The remaining pages expose nothing at all.

And even among the minority who did the hard part, nearly a quarter are missing offers.availability. That single field is what tells an AI agent whether the item is in stock. Without it, a store can be perfectly indexed and still be passed over at the exact moment a shopper’s agent is deciding what to recommend or add to a cart — because the agent cannot confirm the product is buyable.

Why this matters now

Search is shifting from a list of blue links to a synthesized answer, and increasingly to an agent acting on the shopper’s behalf. In that world the winners are not the stores with the prettiest pages — they are the stores whose data is accessible and structured enough for a machine to trust. The scan shows most Magento stores are unprepared on all three fronts at once: some block the crawler, most omit the site-level map, and the great majority ship product pages that machines can’t fully read.

The encouraging half of that story is how fixable it is. None of these gaps require replatforming or a redesign. They require correct robots.txt rules, an llms.txt file, and complete JSON-LD on product pages. Because so few stores have addressed them, the competitive upside for the ones that do is unusually large right now.

The results at a glance

How the angeo.dev suite closes each gap

Every gap the scan surfaced maps to an open-source, MIT-licensed module in the angeo/ suite for Magento 2:

• Blocked AI crawlers → module-robots-txt-aeo manages an explicit, up-to-date AI-bot allowlist with lossless robots.txt round-tripping, so you decide which agents get access instead of a stale template deciding for you.
• Missing llms.txt → module-llms-txt generates and maintains the file automatically from your catalog and CMS content.
• Weak or missing product schema → module-rich-data emits complete JSON-LD Product markup — including offers.availability, shipping details, return policy, and GTIN/MPN — the exact fields the scan found missing.

A Magento AEO checklist

If you’re responsible for a Magento store, start by measuring your own storefront before making any changes, then work through the same signals this scan covered:

• ✓ Verify robots.txt — make sure you aren’t blocking AI crawlers by accident
• ✓ Publish an llms.txt file and keep it in sync with your catalog
• ✓ Emit JSON-LD Product markup on every product page
• ✓ Include offers.availability so agents can confirm stock
• ✓ Validate your schema against Schema.org and Google’s Rich Results Test

The fastest way to see where you stand is module-aeo-audit, which runs this same class of checks against your own Magento store from the command line — including --category and --fail-on-severity flags for CI — so you can measure before and after in under a minute.

Method and limitations

These figures come from a single automated pass over public pages and are reported as-is. Store-level percentages use the 446 confirmed-Magento domains; product-schema percentages use the 223 stores with a confirmed product page. We publish the denominators alongside every figure so the numbers can be checked rather than taken on trust.

A single-pass scan has real limits worth stating plainly. The scan can under-report signals where a page was rate-limited (13 stores hit 429/503), served from a cache or CDN edge, protected by bot mitigation, gated behind login, or rendered client-side in JavaScript so that structured data isn’t present in the initial HTML. Product-page detection is heuristic, which is why we restrict the schema figures to the 223 stores where a product page was positively identified. The results describe what a crawler sees on one visit — a reasonable proxy for what an AI system encounters, but not a substitute for a full per-store audit.