Agent-ready infrastructure for this site.
We transformed our own site into agent-ready infrastructure. Full schema coverage, Markdown twins for every content page, and a Cloudflare edge middleware that serves clean Markdown to AI user agents.
- Stack
- Cloudflare Pages Functions · JSON-LD · schema.org · Claude Code · Model Context Protocol (MCP)
- Time
- ~2 hours (Phase 7 of this site's build)
- Cost
- $0 (Cloudflare Pages free tier)
- Shipped
- Live at winstondigitalmarketing.com
Context: ship it on your own site first.
Most agencies pitch "agent-ready" as a future service. A deck, a workshop, a retainer. We wanted to ship it on our own site first so the pitch comes with a live, inspectable proof point you can verify with one curl command.
This is not a diagram of an architecture we might build for you. It is the architecture running on the page you are reading.
Why not llms.txt.
llms.txt is fine as a convention. It is not a delivery mechanism. If you want AI engines to actually consume clean content, you need a response that you control at the edge.
The approach: three layers.
The system is made of three parts. Each one is independent. Each one is auditable.
- Schema everywhere. Organization on every page. Service on service pages. Article on Playbooks and Receipts. FAQPage on anything with an FAQ. Person on About. The
@graphpattern where multiple types share context. - Parallel Markdown for every content page. Every content page has a companion
.mdfile under/content/with clean heading structure and zero HTML noise. Same URL path, different representation. - Cloudflare Pages Function middleware. Runs at the edge on every request. Detects AI user agents (GPTBot, ClaudeBot, PerplexityBot, CCBot, and others). When matched, serves the
.mdversion withContent-Type: text/markdownand anX-Served-To: ai-agentresponse header.
The core of functions/_middleware.js
const AI_AGENTS = [
/GPTBot/i, /ClaudeBot/i, /PerplexityBot/i, /CCBot/i,
/Google-Extended/i, /anthropic-ai/i, /Bytespider/i,
];
export async function onRequest(context) {
const { request, next } = context;
const ua = request.headers.get("user-agent") || "";
const isAgent = AI_AGENTS.some((rx) => rx.test(ua));
if (isAgent && request.method === "GET") {
const url = new URL(request.url);
const mdPath = toMarkdownPath(url.pathname); // /about/ -> /content/about.md
const md = await context.env.ASSETS.fetch(new Request(url.origin + mdPath));
if (md.ok) {
return new Response(await md.text(), {
headers: {
"Content-Type": "text/markdown; charset=utf-8",
"X-Served-To": "ai-agent",
"Cache-Control": "public, max-age=300",
},
});
}
}
return next();
}
Verify it yourself.
Two commands. One proves the human path. One proves the agent path. If we are lying, you will know.
# Human response
curl -sI https://winstondigitalmarketing.com/
# Agent response
curl -sH "User-Agent: ClaudeBot/1.0" https://winstondigitalmarketing.com/ | head -20
For a human-facing explainer of the same mechanism, see /why-markdown/.
What we shipped.
- JSON-LD schema on all 28 pages of the site
- Matching Markdown versions of every content page under
/content/ - Cloudflare Pages Function middleware at
functions/_middleware.js - A public
/why-markdown/explainer page for humans
Lessons.
- The middleware is lower-effort and higher-value than we expected. About two hours of engineering. Measurable agent traffic within the first day.
- Observability matters. We log the
X-Served-To: ai-agentheader to Cloudflare analytics so we can measure bot traffic by agent family and tune the detection list. - "Agent-ready" and "crawler-ready" are different. Most sites are crawler-ready at best. They serve 200s and valid HTML. Agent-ready means you own the representation the agent consumes, not just the one it can parse.
- Schema is table stakes, not the strategy. It is necessary. It is not sufficient. The middleware is what turns schema into leverage.