Do I need an llms.txt? What is it and is it worth creating one?

Imagine searching for “best Italian restaurants in Munich” – but instead of clicking on links and browsing through different websites, you get an instant personalized answer with specific recommendations, reviews and even reservation advice. Welcome to the new era of AI-powered search!

While for decades we were used to clicking on blue links on Google and then finding the relevant information ourselves, the way people search for information is changing fundamentally. AI systems like ChatGPT, Claude, Perplexity and Google’s AI Overviews no longer just generate a list of websites – they provide direct, comprehensive answers based on content from multiple sources.

This change is not just a technical gimmick. According to Gartner, organic website traffic is expected to decline by 25% by 2026 as AI-generated answers increasingly replace traditional search. Already today, 30% of all search queries and even 75% of problem-solving queries trigger Google AI Overviews.

The central question: How do I optimize my website for AI systems?

This is where a new, experimental approach comes into play: the llms.txt file. But before you hastily create such a file, let’s take an honest look at what’s behind it, whether it works and what alternatives there are.

What is the llms.txt?

An llms.txt file is basically a “signpost” for AI systems – a structured overview of the most valuable content on your website, specially optimized for processing by Large Language Models (LLMs) such as ChatGPT, Claude or Gemini.

Think of llms.txt as a kind of “menu” for AI: Instead of the AI having to laboriously search through your entire website – including navigation, advertising, cookie banners and other “noise” – you show it directly: “Here is my most important, high-quality content in a clean, understandable form.”

Emergence: A standard from the AI community

The llms.txt standard was introduced in September 2024 by Jeremy Howard, the co-founder of Fast.ai and Answer.AI – two renowned organizations in the field of AI research. Howard recognized a fundamental problem: AI systems have limited “context windows” – they cannot process an infinite amount of text at once.

The problem with this is that when an AI system visits your website, it wastes valuable “attention” on HTML code, JavaScript, menus and other technical elements instead of focusing on your actual content.

Purpose: “GPS for AI systems”

Howard describes llms.txt as a “GPS for AI systems” – a precise guide to the most valuable content on your website. The goal is to help AI systems:

• Work more efficiently: Waste less time on irrelevant content
• Provide better answers: Access to your most important, up-to-date information
• To quote more precisely: Correct sources and fewer hallucinations

Technical details: How it works

Implementation is surprisingly simple:

Placement: The file is stored in the root directory of your website, accessible at https://deine-domain.de/llms.txt

Format: Clean Markdown with this basic structure:

# Mein Unternehmen

> Kurze, prägnante Beschreibung deines Projekts oder Unternehmens

## Wichtige Seiten
- [Über uns](https://domain.de/ueber-uns) - Unternehmensinformationen
- [Dienstleistungen](https://domain.de/services) - Unsere Hauptangebote
- [Blog](https://domain.de/blog) - Aktuelle Insights und Expertenwissen

## Ressourcen
- [FAQ](https://domain.de/faq) - Häufig gestellte Fragen
- [Kontakt](https://domain.de/kontakt) - Wie du uns erreichst

Extended version: Optionally, you can create an llms-full.txt that contains not only links, but also complete text versions of your most important content.

The idea behind it is impressively simple: instead of AI systems having to laboriously “crawl” and interpret your website, you serve them the information directly in the form that they can best process.

However, I see a problem here: This llms-full.txt would naturally become very long – and thus the advantage of not “overloading” the LLMs with too much text would be invalid again?

But there’s another catch – and it’s not insignificant…

The sobering reality: lack of support

Now comes the part that will surprise many: Despite the perfectly sensible idea, not a single major AI provider currently officially supports the llms.txt standard.

The list is sobering:

• OpenAI (ChatGPT): No official support
• Google (Gemini, AI Overviews): No announcement
• Anthropic (Claude): No confirmation of use
• Microsoft (Copilot): No integration
• Perplexity: No official statement

This means that even if you create a perfect llms.txt file today, there is no guarantee – not even a hint – that it will actually be used by the AI systems.

Google’s position: A comparison with the outdated keywords meta tag

Google’s skepticism became particularly clear in a statement by John Mueller, one of Google’s best-known SEO employees. He compared llms.txt with the long outdated keywords meta tag – an HTML element that website operators stuffed with search terms in the 1990s until search engines ignored it because it was misused too often.

“It’s super-obvious if you look at your server logs. The consumer LLMs / chatbots (the ones that SEOs want traffic from) will fetch your pages – for training and grounding, but none of them fetch the llms.txt file. Maybe they will tomorrow? Maybe I’ll win in the lottery tomorrow?”

John Mueller

Mueller’s message was clear: AI bots don’t even search for the file.

Community experiences: AI crawler activity in server logs

Theory is one thing – practice is another. SEO experts and website operators have drawn a clear picture:

A Reddit user who had implemented llms.txt on his blog reported: “No AI crawlers have requested the file.” Hosting providers who manage over 20,000 domains confirm: “We don’t see any downloads of llms.txt files by AI agents.”

Even major SEO and content marketing websites have not yet adopted the standard. Companies such as HubSpot, Moz, Ahrefs, SEMrush and Backlinko – all pioneers in digital marketing trends – have not implemented llms.txt files. (as of 22.07.25)

On the other hand, Ryan Howard is currently running an experiment that shows that the AI crawlers have recently been calling up llms.txt more frequently.

The situation is therefore not entirely clear.

What does that mean for you?

These findings do not automatically mean that llms.txt is a bad idea. But they clearly show that it is currently a purely experimental standard with no demonstrable practical impact.

The interesting question is: Should you create an llms.txt file anyway?

Create Llms.txt? Pros and cons

Although llms.txt is currently not officially supported by major AI providers, many website operators are faced with the question: Should I create one anyway? Let’s weigh up the arguments honestly.

Pro arguments: Why it might be worth it

1. first mover advantage: be ready in case standards are adopted

The history of web technologies shows: Early adopters often have advantages. When Google introduced the XML sitemap in 2005, website operators who implemented it immediately benefited. Even robots.txt was originally an unofficial standard before it became the norm.

If OpenAI, Google or other AI providers take over llms.txt after all, you could already be optimally prepared while your competitors are still upgrading.

2. minimal effort: simple creation and maintenance

A basic llms.txt file can be created in 15-30 minutes. All you need is:

• A list of your most important pages
• Short descriptions
• Markdown formatting (which everyone learns quickly)

Tools such as the Yoast SEO plugin or online generators make it even easier – sometimes even fully automatically.

3. structuring exercise: Forces reflection on the most important content

Here’s an often overlooked value: creating an llms.txt forces you to really think things through:

• What content best represents your company?
• What is current and what is outdated?
• How would you explain your website to a stranger in 30 seconds?

This reflection can improve your entire content strategy – regardless of whether AI systems ever read the file.

Contra-arguments

1. no demonstrable effect: currently pure speculation

The toughest argument against llms.txt: there is zero evidence that it works. No success stories, no data, no confirmation from AI vendors. You’re investing time in something that may never deliver value.

2. maintenance effort: regular updates required

Once an llms.txt file has been created, it is useless if it becomes obsolete. You must update it regularly:

• Add new important content
• Remove outdated links
• Customize descriptions
• Repair broken links

This can develop into an annoying, recurring task.

3. opportunity costs: time could be invested in proven SEO measures

The time you spend on llms.txt could be invested in proven strategies:

• Technical SEO: Improve Core Web Vitals
• Content optimization: Revise existing articles
• Strengthen E-E-A-T: Author bios, references, testimonials
• Local SEO: Optimize Google Business Profile

These measures have demonstrable, immediate effects.

4. uncertain future: standard may never prevail

Not every proposed web standard catches on. Internet history is full of good ideas that never became mainstream. llms.txt could be forgotten in a year, while you have invested your time in a “dead horse”.

My pragmatic advice

Create a simple basic version, but don’t invest too much time. Think of it as a low-cost experiment, not a critical SEO measure. In total, creating and uploading the llms.txt will probably not cost you more than an hour – in my opinion, it’s worth a try.

Practical implementation (if you decide to do so)

You have decided to test llms.txt? Then let’s see how to do it right. There are two main ways: manual creation or automated tools.

Manual creation: Full control over your content

A well-structured llms.txt file follows a clear pattern. Here are the most important rules:

1. use a clear hierarchy:

• An H1 headline (your company/website name)
• A concise block quote as a short description
• H2 headings for different categories
• Bullet points for individual links

2. quality before quantity:

• Link a maximum of 10-15 important pages
• Preferably less, but really relevant content
• Each link should have a short, meaningful description

3. clear, natural language:

• Avoid marketing buzzwords
• Write as if you were explaining your website to a colleague
• Descriptions should be 5-15 words long

Example of an optimal llms.txt file

Here is a practical example for a consulting company:

# TRYSEO

> Wir helfen mittelständischen B2B Unternehmen dabei, von potenziellen Kunden gefunden zu werden, und Interessenten in Kunden zu verwandeln.

## Unsere Kernkompetenzen
- [SEO / GEO Beratung](https://www.tryseo.de/seo-beratung/) - Suchmaschinenoptimierung / Generative Engine Optimization 
- [Webentwicklung](https://www.tryseo.de/webdesign/) - Website Erstellung

## Wissensressourcen
- [SEO / GEO Blog](https://www.tryseo.de/blog/) - Aktuelle Insights und Praxistipps
- [Case Studies](https://www.tryseo.de/erfolge/) - Konkrete Projektergebnisse unserer Kunden

## Unternehmensinformationen
- [Über uns](https://www.tryseo.de/ueber-uns/) - Team und Unternehmensphilosophie
- [Kontakt](https://www.tryseo.de/kontakt/) - Kontaktinformation und Terminvereinbarung

Important dos and don’ts:

✅ Do that:

• Use https://-URLs (never http://)
• Test all links before publishing
• Keep descriptions concrete and specific
• Focus on your unique selling points

❌ Avoid that:

• Internal links to imprint or data protection
• Too many links (overwhelms AI systems)
• Promotional language (“best solutions”, “market leader”)
• Outdated or temporary content

Regular maintenance and updates

An llms.txt file is not a “set-and-forget” tool. Schedule this maintenance routine:

Check monthly:

• Are all links still working?
• Are the descriptions still up to date?
• Is there any new important content to add?

Optimize quarterly:

• Are the linked pages still your most important ones?
• Reflect: Would a stranger understand your business through these links?
• Performance analysis: Which pages get the most traffic?

For major website updates:

• Add new main categories
• Remove obsolete areas
• Customize description in block quote

Automated tools: Less effort, more consistency

WordPress: Yoast SEO plugin (since July 2025)

The popular Yoast SEO plugin is the first major tool to integrate llms.txt support. The functions:

Automatic creation:

• Analyzes your WordPress website automatically
• Identifies the most frequently visited and most up-to-date pages
• Generates an updated llms.txt file every week

Manual adjustments:

• You can add or exclude specific pages manually
• Descriptions can be customized
• Integration with other Yoast features (e.g. Primary Category)

Setup in 3 steps:

1. Update Yoast SEO (Version 23.0+)
2. Activate llms.txt under “Yoast > General > Functions”
3. Make adjustments under “Yoast > Tools > File editor”

Online generators: fast and uncomplicated

If you don’t use WordPress, there are several online tools:

llms-txt.io:

• Free tool with intuitive interface
• Enter your most important URLs
• Automatic description generation (editable)
• Download as finished .txt file

Conclusion: A small experiment with great potential

After everything we have discussed about llms.txt, my honest assessment is that it is worth an experiment – but no more.

The llms.txt is currently a standard without any real support. No major AI provider officially uses it, the server logs show no AI crawler activity, and even Google warns against expecting too much. Anyone who creates an llms.txt file today is investing in an uncertain future.

Why I still recommend it

But here’s the trick: the effort is minimal, but the potential is considerable.

A basic llms.txt file costs you a maximum of one hour – once created, a few minutes per month for updates. That’s less time than you spend on an average Instagram post. At the same time, you’re positioning yourself for a possible future in which AI systems actually use these files.

My pragmatic recommendation

Create a simple version if:

• You already have solid SEO basics
• 30-60 minutes time investment is manageable for you
• You like to experiment with new technologies
• You run a well-structured website with high-quality content

Focus on the tried and tested first, if:

• Your website still needs SEO basics
• You have very limited resources
• You prefer to invest in safe strategies

Getting the experiment right

If you decide to do so:

1. Start small: 5-8 of your most important pages are enough
2. Use tools: Online generators or the Yoast plugin save time
3. Set limits: Maximum one hour for creation, 10 minutes per month for updates
4. Don’t expect anything: Think of it as low-cost future insurance, not an SEO magic bullet

Whether llms.txt catches on or not, optimization for AI systems is becoming increasingly important. Today, it’s quotes, statistics and E-E-A-T factors that are proven to work. Tomorrow it could be llms.txt files. The day after tomorrow, something completely different.

My advice: Invest 80% of your SEO time in proven strategies and 20% in GEO measures, as well as forward-looking experiments such as llms.txt. This way you stay on the safe side and don’t miss out on any important developments.

Your experts for the future of search: TRYSEO

While others are still discussing llms.txt, we at SEO & GEO agency TRYSEO are already implementing the scientifically proven GEO strategies that work today:

✅ Generative Engine Optimization (GEO) – optimization for ChatGPT, Perplexity & Co.
✅ E-E-A-T Implementation – expertise and trust for man and machine
✅ Future-proof SEO – classic search engines + AI systems covered