Cloudflare has introduced AI reporting tools that map how automated crawlers visit a site and how often a brand appears in AI-generated answers. The data covers traffic from search and model crawlers, including visits beyond Google and Yandex, giving site owners a view of bots that standard analytics often miss.
What the tools track
The tools identify bots by User-Agent, IP address and behaviour, then link each crawler to the company operating it and its purpose, whether search, model training or data collection. They also separate traffic that produces human visits from crawlers that only gather content, and track brand mentions in AI answers, including citations in ChatGPT and Gemini.
The reporting addresses a gap in search visibility measurement. A page may be used in an AI-generated answer without generating a referral or a link, while competitors may already be cited more often in the same responses.
For SEO specialists, the crawl data can guide markup and metadata adjustments. Crawlers that deliver no results can also be excluded through robots.txt, which reduces server load. For traffic arbitrage and offer teams, separating human visits from model crawlers makes it possible to attribute traffic more accurately and judge which sources meet network requirements.
