> ## Content Index > Fetch the complete content index at: https://getpublii.com/blog/llms.txt > Use this file to discover other available public pages before exploring further. # Should you publish Markdown versions of your web pages for AI? - URL: https://getpublii.com/blog/markdown-for-ai-web-pages.html - Published: 2026-08-14T08:05:56.652Z - Updated: 2026-08-14T09:20:22.343Z - Description: Should you publish Markdown versions of your web pages for AI? Learn what Google, Cloudflare and AI platforms say, how .md files work, and when they make sense. - Author: Bob Mitro - Tags: Publishing For most of the web's history, publishers have focused mainly on two audiences: people reading pages in browsers and search engines crawling them. ![Markdown for AI](https://getpublii.com/blog/media/posts/92/markdown-pages-for-ai-2.webp) AI systems are now becoming another important consumer of web content. ChatGPT search, Claude, Perplexity, coding assistants, research tools and browser agents can all retrieve information from websites in different ways. That raises a practical question for publishers: **should a website keep publishing normal HTML, but also provide a clean Markdown version of each article or page?** For example: ```markdown https://example.com/guides/coffee-brewing/ https://example.com/guides/coffee-brewing.md ``` The content is the same. Only the representation changes. This can easily sound like another "AI SEO" technique, especially now that publishers are being told to optimize everything for AI. The reality is more nuanced. Google explicitly says Markdown files are not needed for its generative search features. OpenAI does not tell publishers to create them, and neither Anthropic nor Perplexity says that publishing `.md` versions will improve visibility in their search products. At the same time, Cloudflare has built Markdown delivery specifically for AI agents, Mintlify provides Markdown versions of documentation pages, and the latest `llms.txt` proposal includes a way for HTML pages to advertise their Markdown alternatives. So the useful question isn't whether Markdown is a new ranking trick. It's whether providing a cleaner representation of existing content makes technical sense for AI systems that choose to use it. ## The idea is fairly simple A typical webpage contains much more than the article itself. Navigation, breadcrumbs, cookie notices, forms, related posts, social buttons, repeated footer links, layout wrappers, styles and scripts may all sit around the actual content. Browsers need this because they have to construct the page that people see and interact with. An AI system interested mainly in the article may have a different task. It has to retrieve the document, identify the useful content and separate that content from the surrounding interface before it can process it further. Modern systems are already capable of doing this with HTML, so Markdown is not necessary for AI to understand a website. It can, however, provide a cleaner version when a client specifically wants a text-oriented representation. Instead of receiving something like: ```html

How to brew better coffee

Published August 14, 2026

Good coffee starts with the right ratio...

Choose your beans

Freshly roasted beans...

``` the same document could be available as: ```markdown # How to brew better coffee - URL: https://example.com/guides/coffee-brewing/ - Published: 2026-08-14T10:00:00.000Z Good coffee starts with the right ratio... ## Choose your beans Freshly roasted beans... ``` The second version contains less presentation markup and less interface content to discard. It doesn't give the AI different information. It simply presents the same information in a more compact form. ## Why Markdown works well for this Markdown wasn't created for artificial intelligence. It has been around since 2004, long before today's AI assistants appeared. Its advantage here is straightforward: Markdown is plain text, but it still preserves useful document structure. Headings remain headings, lists remain lists, links preserve both their label and destination, and code can remain clearly separated from prose. For example: ```markdown ## Installation - Windows - macOS - Linux [Download Publii](https://example.com/download/) ``` Tables, quotations, emphasis and document hierarchy can also survive without carrying all of the markup required to render a complete website. The format itself isn't something invented for the current AI boom either. The `text/markdown` media type was formally registered by the IETF in [RFC 7763](https://datatracker.ietf.org/doc/html/rfc7763) in 2016. For software that ultimately needs textual content rather than a complete visual page, Markdown is therefore a sensible format to work with. What makes the topic more interesting now is that major web infrastructure has started supporting exactly this use case. ## Cloudflare moved the idea into production In February 2026, [Cloudflare introduced Markdown for Agents](https://blog.cloudflare.com/markdown-for-agents/). Sites using Cloudflare can let clients request an existing webpage with an HTTP `Accept` header: ```http GET /article/ HTTP/1.1 Host: example.com Accept: text/markdown ``` Cloudflare can retrieve the original HTML, convert it and return Markdown instead: ```http HTTP/1.1 200 OK Content-Type: text/markdown ``` This is significant because Markdown for agents is no longer only a proposal or an experiment. It is available as part of production web infrastructure. Cloudflare also published numbers showing why it considers the conversion useful. Its announcement article contained about **16,180 tokens as HTML and 3,150 after conversion to Markdown**, roughly an 80% reduction for that particular page. This should not be treated as a universal benchmark, since different websites contain very different amounts of markup, but it demonstrates how large the difference can be. The current [Cloudflare documentation](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) contains another example: ```http x-markdown-tokens: 725 x-original-tokens: 12345 ``` Cloudflare has also said that it has observed coding agents such as Claude Code and OpenCode requesting Markdown through HTTP content negotiation. That is useful evidence because it shows that at least some real agent software is already asking servers for this representation. ## Mintlify follows a similar approach Documentation platform [Mintlify provides Markdown exports](https://www.mintlify.com/docs/ai/markdown-export) for documentation pages as well. A page can be accessed by adding `.md` to its URL: ```markdown /docs/getting-started /docs/getting-started.md ``` Mintlify also supports HTTP content negotiation: ```http GET /docs/getting-started HTTP/1.1 Host: example.com Accept: text/markdown ``` Its documentation presents this as a way to provide clean content to AI tools, LLM integrations and automated workflows. Neither Cloudflare nor Mintlify proves that publishing Markdown improves ranking in ChatGPT, Claude, Google or another AI search system. What they do show is that AI-oriented infrastructure is beginning to treat Markdown as a useful alternate representation of existing web content. ## How this connects with llms.txt If you've read our [complete guide to llms.txt](https://getpublii.com/blog/llms-txt-complete-guide.html), the two ideas fit together naturally, although they solve different problems. An `llms.txt` file can act as a compact index pointing an AI system toward useful resources: ```markdown # Example website > Guides and resources about static website development. ## Guides - [Installing Publii](https://example.com/install.md): Installation instructions - [Creating a website](https://example.com/create-site.md): Getting started guide ``` The Markdown files are the individual documents behind those links. Version 2 of the `llms.txt` proposal, published in August 2026, also addresses an obvious discovery problem: how should software looking at an HTML page know that a Markdown representation exists? The proposal recommends advertising it with: ```html ``` It also introduces `rel="describedby"` for linking a page to the relevant `llms.txt` file. These relationships can be provided as HTML `` elements or through HTTP `Link` headers. The proposal recognizes both: `/page.html.md` and `/page.md` as possible URL patterns for Markdown alternatives. This is useful progress, but `llms.txt` remains a community proposal rather than a universal web standard supported by every major AI provider. A convention can be useful before it becomes universal, but the distinction matters when deciding what publishers should expect from it. ## What Google says about Markdown for AI Google's current position is unusually clear. In its official [guidance for AI features in Search](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), Google says publishers do not need new machine-readable files, AI text files, special markup, `llms.txt` or Markdown to appear in Google Search, including its generative AI features. Google also says maintaining these files for other systems is fine, but they neither help nor harm visibility or rankings in Google Search because Google Search does not use them for that purpose. So there is currently no basis for claiming that creating Markdown versions of articles will make them rank higher in Google's AI results. Normal technical SEO, crawlability, indexability and useful content still matter. This is important because it separates a potentially useful publishing technique from claims that are not supported by the platforms themselves. ## OpenAI, Anthropic and Perplexity don't promise a ranking benefit either The public crawler documentation from other major AI companies doesn't establish Markdown as a ranking signal. OpenAI documents separate roles for [`OAI-SearchBot`, `GPTBot` and `ChatGPT-User`](https://developers.openai.com/api/docs/bots). `OAI-SearchBot`, for example, is used for surfacing websites in ChatGPT search. The documentation explains how publishers can control crawler access, but it does not instruct them to create Markdown copies of their pages or say that `.md` files receive preferential treatment. Anthropic similarly documents [`ClaudeBot`, `Claude-SearchBot` and `Claude-User`](https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler). Its public crawler guidance does not describe Markdown as a visibility or ranking factor. Perplexity documents [`PerplexityBot` and `Perplexity-User`](https://docs.perplexity.ai/docs/resources/perplexity-crawlers), but again, there is no published claim that providing parallel Markdown documents improves ranking. ## Better input is not the same as better discovery This is the key distinction. **Will publishing `/article.md` make an AI search engine discover, cite or rank `/article/` more highly?** We currently don't have reliable evidence that it will. Google explicitly says Markdown doesn't affect Google Search rankings, and the other major AI providers don't publish claims that Markdown receives preferential treatment. There is now some real-world data behind that caution. In 2026, Profound ran a randomized experiment across 381 pages on six websites, comparing pages served to AI bots as HTML with pages served as Markdown. The Markdown group received slightly more bot traffic on average, and ChatGPT-User showed a directional increase of roughly 20%, but the differences were not statistically significant. The experiment therefore did not establish that serving Markdown caused more AI bot traffic. Large citation datasets point in a similar direction. Promptwatch reports that Markdown files accounted for just 0.05% of AI search citations in its dataset. This doesn't prove that Markdown is never used during retrieval or processing, but it does show that `.md` URLs currently make up only a tiny share of the sources cited directly in AI search results. A smaller set of tests by Malte Landwehr found something similar from another angle. Across five websites and nearly 18,000 observed AI citations, sites that offered `.md` copies of their content were cited more than 3,500 times, but none of those citations pointed to the Markdown alternatives. GitHub was a separate case, since Markdown files there are often the normal public documents rather than alternate copies of HTML pages. These results shouldn't be read as proof that Markdown is useless. They support a narrower conclusion: **there is currently no convincing evidence that the format itself creates a discovery, ranking or citation advantage.** The second question is different: **can a Markdown version make a page easier or cheaper for a tool to process when that tool already wants the content?** Here we have much stronger technical evidence. Cloudflare has demonstrated substantial reductions in token count when converting HTML pages to Markdown, and it has observed real coding agents requesting `text/markdown` through HTTP content negotiation. Mintlify also exposes documentation directly as Markdown for AI tools and automated workflows. Those are two different problems. Markdown may provide a cleaner representation and reduce processing overhead without doing anything to improve discovery or ranking. ## HTML should remain the primary document None of this suggests that HTML should be replaced. HTML remains the primary publishing format for the web. It provides semantic structure, navigation, interactive elements, forms, media, accessibility information, metadata, structured data and everything else needed to turn content into an actual website. Modern agents are also not limited to plain-text extraction. Google's own documentation discusses browser agents capable of working with rendered pages, DOM structures and accessibility trees. A good HTML page is therefore already useful to AI systems. Markdown is better viewed as another representation of the same resource: ```markdown /article/ → primary HTML page /article.md → alternate Markdown representation ``` The HTML page remains the document people normally visit. The Markdown file gives clients that prefer a lighter text representation another way to retrieve the same content. ## Keep both versions consistent If a Markdown alternative exists, its substantive content should remain consistent with the original page. A sensible transformation looks like this: ```markdown HTML document ↓ remove presentation and interface noise ↓ preserve meaningful content and context ↓ Markdown document ``` The Markdown version may omit interface-specific instructions that no longer make sense outside the webpage, and it may expose useful metadata associated with the original document. What it should not become is a second article containing different factual claims, hidden promotional instructions or text written only to influence AI systems. In other words, the format can change. The underlying information shouldn't. ## Which pages are good candidates? There is no official platform rule defining which URL types should have Markdown alternatives, so this part is a practical publishing recommendation rather than an established AI standard. - **Posts and articles:** Blog posts, tutorials, guides, reviews, news articles, essays and knowledge-base entries are strong candidates. They usually have a clear title, useful structure and a substantial body of information that makes sense outside the surrounding website interface. - **Content-rich pages:** About pages, product explanations, service pages, policies, FAQs and substantial landing pages can also be useful. Their value depends on the content rather than on whether the CMS labels them as a "page". - **Documentation:** Documentation is probably the clearest use case. Mintlify already exposes documentation as Markdown, Cloudflare supports Markdown delivery for agents, and coding assistants frequently need individual guides, API references and configuration instructions. - **Categories, tags and other collection pages:** Taxonomy archives are usually less useful when they contain little more than a heading and a list of excerpts linking to documents that already exist elsewhere. The same applies to author listings, pagination and similar collection views. If one of these pages contains substantial unique information, there may be a case for exposing it separately, but there is little reason to duplicate it merely because the URL exists. - **Search results:** Search results are primarily navigational states rather than standalone documents, so they are generally poor candidates. - **The homepage:** This depends on how the site is built. A homepage containing substantial information about a product, organization or project may work well as Markdown. A homepage made mostly of visual cards linking elsewhere provides less value as a standalone text document. A practical approach is to generate Markdown for pages that contain useful standalone information rather than mirroring every possible URL on the site. ## What should the Markdown file contain? The goal isn't to reduce everything to plain paragraphs. Structure still matters. A useful document might begin with: ```markdown # How to build a static website - URL: https://example.com/how-to-build-static-website/ - Published: 2026-08-14T10:00:00.000Z - Updated: 2026-08-14T12:00:00.000Z - Description: A practical guide to building and publishing a static website. - Author: Jane Smith - Tags: Static sites, Web development Static websites have changed considerably... ``` The main content can preserve: - headings - paragraphs - lists - links - quotations - useful image descriptions - tables - code blocks - relevant document metadata Repeated navigation, cookie UI, social sharing controls, layout wrappers and unrelated recommendations usually add much less value to a text-oriented representation. ## How can the Markdown version be exposed? There is no single mechanism used universally today. Several approaches are appearing alongside each other. ### A predictable .md URL For a static website, the simplest option is often: ```markdown /post/ → /post.md ``` The Markdown file is just another static resource. It can be cached and served by ordinary hosting without requiring special server logic. ### An alternate link in HTML The `llms.txt` v2 proposal recommends declaring the Markdown representation explicitly: ```html ``` This allows software inspecting the HTML document to discover the alternate representation instead of guessing whether a corresponding `.md` URL exists. Support should not be assumed to be universal, but the relationship itself is clear and inexpensive to provide. ### Links from llms.txt If the website publishes `llms.txt`, entries in that file can point directly to Markdown versions where available. This lets the index and the individual text documents complement each other. ### HTTP content negotiation Cloudflare shows another approach: ```http GET /post/ HTTP/1.1 Host: example.com Accept: text/markdown ``` The URL stays the same while the returned representation changes according to the requested media type. This avoids creating a second public URL, but it requires support from the application, server or CDN. For a static site generator, producing a separate `.md` file is often much easier. ## What about SEO and Google Search? Publishing the same content as HTML and Markdown naturally raises an SEO question: could the additional `.md` file affect how Google treats the original page? According to Google's current guidance, publishers do not need Markdown, `llms.txt`, AI text files or other special machine-readable files to appear in Google Search or its generative AI features. Google states that maintaining such files for other systems will **neither help nor harm a site's visibility or rankings in Google Search**. That means a Markdown version should not be treated as an additional SEO page or as a new ranking signal. Its purpose is to provide another representation of content that already exists in HTML. The HTML document should remain the primary version, with its existing canonical URL, title, meta description, robots settings and structured data left unchanged. A Markdown document may include useful context such as the original HTML URL, publication and update dates, description, author and tags, but this information does not replace the SEO metadata of the original page. Google may still discover, crawl and index text-based resources outside normal HTML pages, so publishers should not assume that a public `.md` file is invisible to Search. Google also notes that duplicate or alternative URLs can consume crawling resources, which matters more on large sites where many additional URLs are generated. Google's current guidance gives no indication that publishing a faithful Markdown alternative should cause the original HTML page to lose rankings. At the same time, there is little reason to present the Markdown version as another page intended to compete for search visibility. For that reason, I would keep the XML sitemap focused on the primary HTML URLs and preserve existing indexing decisions such as `noindex`. The Markdown file should remain an alternate representation of the document rather than a separate SEO landing page. This is also the approach taken by Publii's Markdown for AI plugin. Generated `.md` files are not added to the website sitemap, the existing HTML SEO settings remain unchanged, and content marked `noindex` is excluded from Markdown generation by default. If you deploy Markdown versions across an existing website, it is still sensible to monitor Google Search Console afterwards. Search engines can change how they discover and handle alternative content formats over time, and real indexing behavior on your own site is ultimately more useful than assuming every implementation will behave identically. ## Not every website needs Markdown versions A small restaurant website or portfolio doesn't suddenly become inaccessible to AI because it doesn't publish `.md` files. A well-structured HTML site remains perfectly valid, and Google explicitly says special AI files are not required for its generative Search features. Markdown is better understood as an optional publishing layer. Its value depends on the kind of content you publish and, perhaps more importantly, on how much work it takes to maintain. ## Automatic generation changes the calculation Manually maintaining two versions of every article would be difficult to justify. A correction would need to be made twice, links could drift out of sync, and metadata could easily become inconsistent. There is currently no evidence of a visibility benefit large enough to justify that editorial overhead. The situation is different when both files are generated automatically from the same source content. The author still writes and updates one document, while the publishing system creates the HTML page and the Markdown representation during the same build process. For a content-heavy site, the additional cost can then become very small. At that point the question is less about whether Markdown will increase AI traffic and more about whether it makes sense to expose a representation that some agents and AI infrastructure already know how to request. For documentation, technical blogs, knowledge bases and other content-heavy websites, I think it increasingly does. ## If you're using Publii CMS This is the approach used by our [Markdown for AI](https://marketplace.getpublii.com/plugins/markdown-for-ai/) plugin for Publii. The plugin doesn't replace the existing HTML page and doesn't create a second editorial workflow. You continue writing and updating content in Publii as usual. During site generation, the plugin can create a companion `.md` file from the same source content. For example: ```markdown /post/ → /post.md /post.html → /post.md /page/about/ → /page/about.md /page/about.html → /page/about.md ``` The plugin deliberately focuses on individual posts and pages rather than generating Markdown for every URL on the site. Tag listings, author listings, archives, pagination, 404 pages and other collection views are not duplicated as Markdown documents. This follows the same principle discussed earlier in this article: Markdown is most useful for documents containing substantial standalone content, not simply for every URL a website can generate. The generated files preserve useful document structure such as headings, paragraphs, lists, links, images, tables, quotations and code blocks. Depending on the source content, they can also include context such as the original HTML URL, publication and update dates, description, author and tags. When a Markdown file is successfully generated, the corresponding HTML page can advertise it with: ```html ``` The HTML page remains the primary version. Its canonical URL, title, meta description, robots settings and structured data are left unchanged, and generated Markdown files are not added to the website sitemap. Content marked `noindex` is excluded from Markdown generation by default. If an existing post or page is later changed to `noindex`, its generated Markdown counterpart is removed during the next site generation and the HTML page stops advertising the alternate version. ## Markdown for AI and LLMS.txt Generator work together automatically [Markdown for AI](https://marketplace.getpublii.com/plugins/markdown-for-ai/) and Publii's [LLMS.txt Generator](https://marketplace.getpublii.com/plugins/llms-txt-generator/) can be used independently, but when both are enabled they detect each other and work together automatically. LLMS.txt Generator can prefer Markdown links for eligible posts and pages. When a generated `.md` file exists, the corresponding entry in `llms.txt` can point directly to it: ```markdown ## Guides - [Installing Publii](https://example.com/install.md): Installation instructions - [Creating a website](https://example.com/create-site.md): Getting started guide ``` This isn't a blind URL replacement. The plugin checks whether the Markdown file was actually generated. If it doesn't exist because the content was excluded, marked `noindex` or otherwise wasn't generated, the original HTML URL remains in `llms.txt`. That means a single index can contain both Markdown and HTML destinations when needed: ```markdown ## Guides - [Installing Publii](https://example.com/install.md): Installation instructions - [About our project](https://example.com/about/): Learn more about the project ``` The integration also works in the other direction. When Markdown for AI detects an active LLMS.txt Generator, generated Markdown documents can include a short reference back to the site's main content index: ```markdown > ## Content Index > Fetch the complete content index at: https://example.com/llms.txt > Use this file to discover other available public pages before exploring further. # Post title ``` This creates a simple relationship between discovery and content. The `llms.txt` file can help a tool find relevant documents, while an individual Markdown document can point back to the wider site index. No manual connection between the two plugins is required, and disabling either plugin doesn't prevent the other from working normally. None of this changes the SEO role of the original HTML pages or guarantees that an AI platform will crawl, cite or rank the site more often. The purpose is more practical: provide a structured content index and clean document representations to systems that choose to use them. ## The bigger picture The web isn't abandoning HTML. Browsers still need it, search engines understand it, and increasingly capable agents can work with it directly. What is changing is the range of software consuming web content. Some tools operate on rendered pages, while others benefit from a lightweight text representation that removes much of the surrounding interface. Markdown won't replace HTML and it isn't a requirement for AI visibility. But we can now point to real infrastructure, including Cloudflare and Mintlify, that already treats it as a useful representation for machine consumption. For publishers who can generate it automatically, that makes it worth considering. **HTML for the Web. Markdown when a machine wants the document without the website wrapped around it.**