Cloudflare Launches Website Scanner for AI Agent Readiness Assessment
Cloudflare has introduced a tool that evaluates websites for compatibility with AI agents, checking compliance with emerging technical standards. The scanner assesses five categories: discoverability through robots.txt and sitemaps, content accessibility via Markdown negotiation, bot access control with AI-specific rules, protocol discovery for MCP servers and OAuth, and commerce capabilities using x402, UCP, and ACP protocols. Recommendations include publishing valid robots.txt files with AI bot directives and ensuring homepage metadata contains useful discovery headers. Developers can consult Cloudflare's documentation to build agents capable of web browsing, interaction, and transactions. The tool provides actionable instructions for coding agents like Cursor, Claude Code, Windsurf, and Copilot to enhance site readiness. A dedicated website at isitagentready.com hosts the scanning service.
Key facts
- Cloudflare offers a website scanner for AI agent readiness
- Tool checks compliance with emerging AI agent standards
- Five assessment categories: discoverability, content accessibility, bot access control, protocol discovery, commerce
- Recommends valid robots.txt with AI bot rules and sitemap directives
- Suggests homepage metadata include discovery headers
- Cloudflare documentation available for building AI agents
- Instructions provided for coding agents like Cursor and Claude Code
- Scanner available at isitagentready.com
Entities
Institutions
- Cloudflare