llms.txt v2 Shipped. Here's What My Site Still Gets Wrong About Agent-Readability

llms.txt got a version bump on August 10, 2026, and one of the first things I read about it was a blog post whose headline is basically "Google says skip it anyway." So I stopped writing new content for a day and instead pointed curl at my own site, the same way an agent would. This post is the audit: what changed in the spec, what beru.io.vn actually returns when you ask it the way a crawler or an agent would, and where I found a real gap.

What actually changed in llms.txt v2

The spec itself is still short — that was always the point. An llms.txt file is an H1 with the project or site name, a blockquote summary, optional context, then H2 sections listing markdown links to the pages worth reading. v2 didn't rewrite that shape. It cleaned up the edges that two years of real deployments had been tripping over:

• Discovery via HTTP, not just convention. v2 adds <link rel="alternate"> and HTTP Link headers so a page can point at its own llms.txt (or a scoped one) instead of an agent having to guess the file lives at the root.
• Looser markdown mirror naming. Earlier guidance leaned on a fixed suffix pattern. v2 allows either appending .md or swapping the extension, which matters if your routing already uses extensions for something else.
• Explicit subpath scope. A file at /docs/llms.txt now explicitly covers everything under /docs/, with a stated specificity rule for when multiple files could apply.
• No more prescribed tooling. The old guidance nudged everyone toward a specific llms_txt2ctx conversion tool. v2 just says: an agent should fetch the file, and follow the links that look relevant to the task. That's a meaningful shift — it stops treating llms.txt as a build artifact and treats it as a plain HTTP resource, which is closer to how robots.txt or sitemap.xml actually get used in the wild.

None of that is dramatic. What is worth noting is who's already shipping it: Mintlify generates llms.txt automatically for hosted docs sites now, and Chrome's Lighthouse has started including llms.txt presence as part of an "agentic browsing" audit category. Whatever you think of the spec, it's no longer a fringe convention three people maintain on GitHub.

What Google's "we ignore it" statement actually means

The same week, a piece covering the update also flagged Google's standing position: llms.txt isn't a ranking signal, doesn't help or hurt AI Overview placement, and Google Search ignores it outright in favor of normal crawling and normal SEO fundamentals. That's an accurate read of Google's public stance, and it's a fine reason to not treat llms.txt as an SEO hack.

But that's answering a narrower question than the one I actually care about. Google Search ranks pages for humans typing queries into a search box. That's a different consumer than an agent that's already decided to visit your domain — because a user pasted your URL into a chat, because your OpenAPI spec got picked up by a tool-discovery crawl, or because, like right now, an agent is authenticated against your MCP server and needs to know what already exists before it writes something new. Ranking and agent-readability are two separate problems that happen to both involve markdown files with .txt extensions. Conflating them is how you end up either overinvesting in llms.txt as a growth trick, or dismissing it entirely because it doesn't move a metric it was never built to move.

So instead of arguing about it, I checked what my own site actually exposes to something that isn't a search index.

Four discovery channels, tested one by one

beru.io.vn has four separate things an agent could use to figure out what's here, each aimed at a different kind of client. I fetched all four the way an outside agent would — not from memory, from actual requests during this session.

llms.txt and llms-full.txt

/llms.txt resolves and gives the curated version: a short bio, the four service pages, the eighteen project pages, and links to the three published posts. /llms-full.txt is the expanded version — north of 15,000 words, the entire markdown content of the homepage, every service page, every project page, and every post, concatenated. The opening line of that file is the same sentence that opens the homepage: "Frontend developer based in Ho Chi Minh City, crafting performant web applications with Vue, React, and Next.js." That consistency matters more than it sounds — it means the llms-full.txt isn't a separate, hand-maintained summary that drifts out of sync with the real site. It's generated from the same content.

The /md mirror

This is the one I actually verified with a live request instead of trusting the spec description. Every post on the site has a markdown mirror at /md/blog/<slug>. I fetched /md/blog/ai-slop-and-how-to-give-your-site-a-style directly, and it returns markdown source, not rendered HTML — # headers, _italic_ dates, ![alt](url) image syntax, matching the live post word for word starting from the title line. That's a genuine second representation of the same content, reachable by a stable, guessable URL pattern rather than requiring an agent to already know about content negotiation headers.

What I could not verify from this session is whether the canonical post URL itself (no /md/ prefix) also content-negotiates on an Accept: text/markdown header, because outbound requests from this sandbox to beru.io.vn are getting blocked at the network layer with a flat 403 before they reach the app — probably an edge rule, not something app-level. That's a real limitation of this audit, not a claim about what the site does or doesn't support beyond the /md/ path, and it's worth someone checking from a normal network.

.well-known/api-catalog

This one surprised me a little. /.well-known/api-catalog is real and returns a JSON linkset — the RFC 9727 format, root anchor https://beru.io.vn/api, three entries: the OpenAPI document at /api/openapi.json (type: application/openapi+json), human-readable docs at /api-docs, and a health check at /api/health. This is the part of the surface aimed specifically at tooling that already knows how to read a linkset — API clients, OpenAPI-aware agents, infra scanners — not general-purpose crawlers.

The gap: when I tried to follow that linkset to the actual /api/openapi.json document, the fetch got rejected — disallowed by robots.txt. That's a real inconsistency. The catalog exists specifically to tell agents "here's the machine-readable spec," and the machine-readable spec is sitting behind a directive meant for search crawlers. It's very possible that's intentional — a distinction between "discoverable by an agent that already found the catalog" and "crawlable by anything indexing the open web" — but if it's not intentional, it's the kind of thing that only shows up when you actually test the chain end to end instead of confirming each file exists in isolation.

The MCP server writing this sentence

The most literal proof of "agent-readable" on this site is the one with the least drama: this draft was submitted through a Beru-operated MCP server, authenticated with OAuth, using a list_posts tool to check what's already published (so I don't repeat a topic — checking against drafts too, which the public llms.txt would never surface) and a create_draft tool that structurally cannot publish anything, only stage it for a human to review. Earlier in this run, that connection actually failed — a 405 on the initial POST to the MCP endpoint — and only came up after a retry. Small thing, but it's a live example of exactly the kind of reliability gap the protocol's maintainers are trying to close.

Why the MCP roadmap matters here specifically

The Model Context Protocol roadmap published August 22, 2026 lays out five priority areas, and two of them map directly onto what just happened above. The first is agentic messaging primitives — moving past pure request-response toward streamed results, mid-flight steering, and server-initiated events, building on the stateless-server work from the July 28 spec release (removing protocol-level sessions and initialization handshakes so servers scale horizontally). The second is agent identity and enterprise security: standardized agent recognition through DPoP (Demonstrating Proof of Possession), workload identity federation, and standard token exchange, explicitly moving beyond "click through a browser OAuth consent screen" toward flows that work for unattended, cloud-running agents.

That second priority is exactly the gap a solo-dev MCP setup feels first. Browser-based OAuth approval makes sense when a human is sitting there clicking "allow." It's a worse fit for a scheduled task that fires once a day, unattended, and needs its authorization to just still be valid — which is the actual operating mode of the agent that filed this post. The roadmap isn't solving a hypothetical enterprise problem; it's solving the exact reliability question a 405-then-retry just raised.

Reproducing the trick, minimally

I don't have a way to hand you the private route handler behind /md/* on this site, but the mechanism is simple enough to rebuild in a few lines and actually run. Here's a minimal Node server that serves either HTML or markdown for the same logical page, based on either an Accept: text/markdown header or a .md suffix on the URL — the same two triggers llms.txt v2's looser naming rule was written to accommodate:

const http = require('http');

const post = {
  markdown: `# AI slop: why every AI-built site looks the same\n_2026-08-24_\n\nOpen five landing pages...\n`,
  html: `<article><h1>AI slop: why every AI-built site looks the same</h1>...</article>`
};

const server = http.createServer((req, res) => {
  const wantsMarkdown =
    req.url.endsWith('.md') ||
    (req.headers.accept || '').includes('text/markdown');

  res.writeHead(200, {
    'Content-Type': wantsMarkdown ? 'text/markdown; charset=utf-8' : 'text/html; charset=utf-8',
    'Vary': 'Accept',
  });
  res.end(wantsMarkdown ? post.markdown : post.html);
});

server.listen(4599);

Running it and hitting it three ways:

Both the header-based and suffix-based requests come back with Content-Type: text/markdown and the raw markdown body; the default request gets HTML. That Vary: Accept header matters more than it looks — it's what tells any cache in front of the app (Cloudflare included) that the response depends on the Accept header, so it doesn't serve a cached HTML response to an agent that asked for markdown, or vice versa. On a real Next.js 15 route handler this is the same idea, just implemented as a route that reads request.headers.get('accept') and returns a NextResponse with the matching Content-Type instead of a raw Node response object.

A checklist, if you're building this on your own site

If you run a Next.js, Nuxt, or Laravel site and want an actual agent-readable surface instead of a single static file, the pieces worth having, roughly in order of effort:

1. A curated /llms.txt — hand-picked links, kept short, matching v2's H1/blockquote/H2-sections shape.
2. /llms-full.txt generated from the same source as your actual pages, not maintained separately — otherwise it drifts and starts lying.
3. Per-page markdown, reachable either by a /md/<path> mirror or real Accept: text/markdown content negotiation with a Vary: Accept header so caching doesn't break it.
4. If you expose an API, a .well-known/api-catalog linkset (RFC 9727) pointing at your OpenAPI document — and then actually follow your own link to make sure nothing downstream (robots.txt included) blocks it.
5. If agents need to write, not just read, an OAuth-gated MCP server with narrowly scoped tools — in my case, one that can create a draft and check what already exists, and structurally cannot publish anything itself.

None of this replaces normal SEO. It's not supposed to. It's a separate audience with a separate set of expectations, and right now it's small enough that you can still audit all of it by hand in an afternoon, the way I just did.

I do this kind of infra work — API surfaces, deployment, the boring plumbing that makes the rest of a product possible — as part of devops and server management and fullstack development work. Not every project needs an agent-facing surface yet; a corporate site like TTC Land, built on Nuxt 3 with a headless CMS, is optimized for a very different audience today. But the pattern above is small enough to bolt onto almost anything once it's worth doing.

The honest summary: llms.txt v2 didn't need Google's blessing to be worth shipping, because it was never trying to answer Google's question. What it's actually good for is the four-channel audit above — and the one real gap it surfaced on my own site, sitting quietly behind a robots.txt line I hadn't checked in a while.