Reference

AI crawlers

Decide whether AI tools can read your blog, and whether they should find it easy.

AI and your blog are really two separate questions, and Bloggrify keeps them separate:

Do you want AI crawlers to use your content at all? That is a robots.txt question.
Do you want to make your content easy for AI tools to read? That is what llms.txt is for.

They are independent, and answering "no" to the first makes the second pointless.

Both live under the seo.ai key of your app.config.ts:

app.config.ts

export default defineAppConfig({
  seo: {
    ai: {
      llms: false,
      allowCrawlers: true,
    },
  },
})

Blocking AI crawlers

Set allowCrawlers to false to ask AI crawlers to leave your content alone:

app.config.ts

export default defineAppConfig({
  seo: {
    ai: {
      allowCrawlers: false,
    },
  },
})

This adds a Disallow: / group to your robots.txt covering the AI crawlers of the major vendors: GPTBot and OAI-SearchBot (OpenAI), ClaudeBot (Anthropic), Google-Extended and Google-NotebookLM (Google), meta-externalagent (Meta), Amazonbot and bedrockbot (Amazon), PerplexityBot, MistralAI-User, DeepSeekBot, Bytespider, CCBot (Common Crawl, which feeds many training sets), and around thirty others. The names follow the ai.robots.txt reference list.

Your blog stays indexed in Google and Bing: search crawlers are deliberately left alone. Google-Extended controls Gemini training and is distinct from Googlebot, so you refuse the former while keeping the latter. For the same reason Applebot (Siri, Spotlight) and facebookexternalhit (Open Graph link previews) are not blocked, since blocking them would break features you probably want. Only Applebot-Extended, Apple's AI-training agent, is.

Bloggrify ships a curated list, not every scraper in existence. If you want to go further, add your own group in your nuxt.config.ts:

nuxt.config.ts

export default defineNuxtConfig({
  extends: ['@bloggrify/core'],
  robots: {
    groups: [
      {
        userAgent: ['SomeOtherBot'],
        disallow: ['/'],
      },
    ],
  },
})

This is a polite request, not a protection. robots.txt is voluntary: well-behaved crawlers honour it, others have been caught ignoring it. Anything published on a static, public blog can be copied. If you need real enforcement, block at the network level instead, for example with Cloudflare's "Block AI Scrapers" toggle. That happens at your host, outside of Bloggrify.

Publishing llms.txt

Set llms to true to publish a /llms.txt route: a markdown index of your posts, with a title, a description and a link for each one. See llmstxt.org for the format.

app.config.ts

export default defineAppConfig({
  seo: {
    ai: {
      llms: true,
    },
  },
})

The file is generated at build time, alongside sitemap.xml and rss.xml. Drafts and hidden posts are excluded. When llms is false, which is the default, the route is not generated at all.

llms.txt is an invitation, not a restriction. It does not tell crawlers what they may do, it only makes your content easier to consume. If you want AI tools to stay away, allowCrawlers: false is the option you are looking for, and enabling both at once is contradictory. Bloggrify prints a warning at build time if you do.

Is it worth enabling?

Be aware that the value is speculative today. No major AI provider has confirmed that its crawlers read llms.txt, and Google has publicly compared it to the old keywords meta tag. The case that does work is a human pasting your URL into a chat: the tool fetches the page, and clean markdown beats HTML full of navigation and scripts.

For a documentation site, that is a real win. For a blog, where each post is already a clean page, the gain is smaller. It costs one small file, so enable it if you like, but do not expect traffic from it.

Indexing

Whether search engines index your blog at all is a separate setting, documented in Robots.txt.

Edit this pageorReport an issue

Analytics

Learn how to add analytics to your blog.

Theming

Learn how to change the look and feel of your blog.