How I Added llms.txt to My Astro Blog

TLDR

I created an endpoint in my Astro blog that outputs all posts in plain text format. This lets me copy my entire blog with one click and paste it into any LLM with adequate context window. The setup uses TypeScript and Astro’s API routes, making it work with any Astro content collection.

Why I Built This

I wanted a quick way to ask AI models questions about my own blog content. Copying posts one by one is slow. With this solution, I can give any LLM all my blog posts at once.

How It Works

The solution creates a special endpoint that:

1. Gets all blog posts
2. Converts them to plain text
3. Formats them with basic metadata
4. Outputs everything as one big text file

Setting Up the File

First, I created a new TypeScript file in my Astro pages directory:

// src/pages/llms.txt.ts
import type { type APIRoute<APIProps extends Record<string, any> = Record<string, any>, APIParams extends Record<string, string | undefined> = Record<string, string | undefined>> = (context: APIContext<APIProps, APIParams>) => Response | Promise<Response>APIRoute } from "astro";
import { import getCollectiongetCollection } from "astro:content";

// Function to extract the frontmatter as text
const const extractFrontmatter: (content: string) => stringextractFrontmatter = (content: stringcontent: string): string => {
  const const frontmatterMatch: RegExpMatchArray | nullfrontmatterMatch = content: stringcontent.
String.match(matcher: {
    [Symbol.match](string: string): RegExpMatchArray | null;
}): RegExpMatchArray | null (+1 overload)
Matches a string or an object that supports being matched against, and returns an array
containing the results of that search, or null if no matches are found.
@parammatcher An object that supports being matched against.match(/^---\n([\s\S]*?)\n---/);
  return const frontmatterMatch: RegExpMatchArray | nullfrontmatterMatch ? const frontmatterMatch: RegExpMatchArrayfrontmatterMatch[1] : '';
};

// Function to clean content while keeping frontmatter
const const cleanContent: (content: string) => stringcleanContent = (content: stringcontent: string): string => {
  // Extract the frontmatter as text
  const const frontmatterText: stringfrontmatterText = const extractFrontmatter: (content: string) => stringextractFrontmatter(content: stringcontent);
  
  // Remove the frontmatter delimiters
  let let cleanedContent: stringcleanedContent = content: stringcontent.
String.replace(searchValue: {
    [Symbol.replace](string: string, replaceValue: string): string;
}, replaceValue: string): string (+3 overloads)
Passes a string and 
{@linkcode 
replaceValue
}
 to the `[Symbol.replace]` method on 
{@linkcode 
searchValue
}
. This method is expected to implement its own replacement algorithm.
@paramsearchValue An object that supports searching for and replacing matches within a string.@paramreplaceValue The replacement text.replace(/^---\n[\s\S]*?\n---/, '');
  
  // Clean up MDX-specific imports
  let cleanedContent: stringcleanedContent = let cleanedContent: stringcleanedContent.
String.replace(searchValue: {
    [Symbol.replace](string: string, replaceValue: string): string;
}, replaceValue: string): string (+3 overloads)
Passes a string and 
{@linkcode 
replaceValue
}
 to the `[Symbol.replace]` method on 
{@linkcode 
searchValue
}
. This method is expected to implement its own replacement algorithm.
@paramsearchValue An object that supports searching for and replacing matches within a string.@paramreplaceValue The replacement text.replace(/import\s+.*\s+from\s+['"].*['"];?\s*/g, '');
  
  // Remove MDX component declarations
  let cleanedContent: stringcleanedContent = let cleanedContent: stringcleanedContent.
String.replace(searchValue: {
    [Symbol.replace](string: string, replaceValue: string): string;
}, replaceValue: string): string (+3 overloads)
Passes a string and 
{@linkcode 
replaceValue
}
 to the `[Symbol.replace]` method on 
{@linkcode 
searchValue
}
. This method is expected to implement its own replacement algorithm.
@paramsearchValue An object that supports searching for and replacing matches within a string.@paramreplaceValue The replacement text.replace(/<\w+\s+.*?\/>/g, '');
  
  // Remove Shiki Twoslash syntax like // @noErrors
  let cleanedContent: stringcleanedContent = let cleanedContent: stringcleanedContent.
String.replace(searchValue: {
    [Symbol.replace](string: string, replaceValue: string): string;
}, replaceValue: string): string (+3 overloads)
Passes a string and 
{@linkcode 
replaceValue
}
 to the `[Symbol.replace]` method on 
{@linkcode 
searchValue
}
. This method is expected to implement its own replacement algorithm.
@paramsearchValue An object that supports searching for and replacing matches within a string.@paramreplaceValue The replacement text.replace(/\/\/\s*@noErrors/g, '');
  let cleanedContent: stringcleanedContent = let cleanedContent: stringcleanedContent.
String.replace(searchValue: {
    [Symbol.replace](string: string, replaceValue: string): string;
}, replaceValue: string): string (+3 overloads)
Passes a string and 
{@linkcode 
replaceValue
}
 to the `[Symbol.replace]` method on 
{@linkcode 
searchValue
}
. This method is expected to implement its own replacement algorithm.
@paramsearchValue An object that supports searching for and replacing matches within a string.@paramreplaceValue The replacement text.replace(/\/\/\s*@(.*?)$/gm, ''); // Remove other Shiki Twoslash directives
  
  // Clean up multiple newlines
  let cleanedContent: stringcleanedContent = let cleanedContent: stringcleanedContent.
String.replace(searchValue: {
    [Symbol.replace](string: string, replaceValue: string): string;
}, replaceValue: string): string (+3 overloads)
Passes a string and 
{@linkcode 
replaceValue
}
 to the `[Symbol.replace]` method on 
{@linkcode 
searchValue
}
. This method is expected to implement its own replacement algorithm.
@paramsearchValue An object that supports searching for and replacing matches within a string.@paramreplaceValue The replacement text.replace(/\n\s*\n\s*\n/g, '\n\n');
  
  // Return the frontmatter as text, followed by the cleaned content
  return const frontmatterText: stringfrontmatterText + '\n\n' + let cleanedContent: stringcleanedContent.String.trim(): string
Removes the leading and trailing white space and line terminator characters from a string.trim();
};

export const const GET: APIRouteGET: type APIRoute<APIProps extends Record<string, any> = Record<string, any>, APIParams extends Record<string, string | undefined> = Record<string, string | undefined>> = (context: APIContext<APIProps, APIParams>) => Response | Promise<Response>APIRoute = async () => {
  try {
    // Get all blog posts sorted by date (newest first)
    const const posts: anyposts = await import getCollectiongetCollection('blog', ({ data: anydata }) => !data: anydata.draft);
    const const sortedPosts: anysortedPosts = const posts: anyposts.sort((a: anya, b: anyb) => 
      new 
var Date: DateConstructor
new (value: number | string | Date) => Date (+4 overloads)Date(b: anyb.data.pubDatetime).Date.valueOf(): number
Returns the stored time value in milliseconds since midnight, January 1, 1970 UTC.valueOf() - new 
var Date: DateConstructor
new (value: number | string | Date) => Date (+4 overloads)Date(a: anya.data.pubDatetime).Date.valueOf(): number
Returns the stored time value in milliseconds since midnight, January 1, 1970 UTC.valueOf()
    );

    // Generate the content
    let let llmsContent: stringllmsContent = '';

    for (const const post: anypost of const sortedPosts: anysortedPosts) {
      // Add post metadata in the format similar to the example
      let llmsContent: stringllmsContent += `--- title: ${const post: anypost.data.title} description: ${const post: anypost.data.description} tags: [${const post: anypost.data.tags.map(tag: anytag => `'${tag: anytag}'`).join(', ')}] ---\n\n`;
      
      // Add the post title as a heading
      let llmsContent: stringllmsContent += `# ${const post: anypost.data.title}\n\n`;
      
      // Process the content, keeping frontmatter as text
      const const processedContent: stringprocessedContent = const cleanContent: (content: string) => stringcleanContent(const post: anypost.body);
      let llmsContent: stringllmsContent += const processedContent: stringprocessedContent + '\n\n';
      
      // Add separator between posts
      let llmsContent: stringllmsContent += '---\n\n';
    }

    // Return the response as plain text
    return new var Response: new (body?: BodyInit | null, init?: ResponseInit) => Response
This Fetch API interface represents the response to a request.

[MDN Reference](https://developer.mozilla.org/docs/Web/API/Response)Response(let llmsContent: stringllmsContent, {
      ResponseInit.headers?: HeadersInit | undefinedheaders: { "Content-Type": "text/plain; charset=utf-8" },
    });
  } catch (function (local var) error: unknownerror) {
    var console: Console
The `console` module provides a simple debugging console that is similar to the
JavaScript console mechanism provided by web browsers.

The module exports two specific components:

* A `Console` class with methods such as `console.log()`, `console.error()` and `console.warn()` that can be used to write to any Node.js stream.
* A global `console` instance configured to write to [`process.stdout`](https://nodejs.org/docs/latest-v22.x/api/process.html#processstdout) and
[`process.stderr`](https://nodejs.org/docs/latest-v22.x/api/process.html#processstderr). The global `console` can be used without importing the `node:console` module.

_**Warning**_: The global console object's methods are neither consistently
synchronous like the browser APIs they resemble, nor are they consistently
asynchronous like all other Node.js streams. See the [`note on process I/O`](https://nodejs.org/docs/latest-v22.x/api/process.html#a-note-on-process-io) for
more information.

Example using the global `console`:

```js
console.log('hello world');
// Prints: hello world, to stdout
console.log('hello %s', 'world');
// Prints: hello world, to stdout
console.error(new Error('Whoops, something bad happened'));
// Prints error message and stack trace to stderr:
//   Error: Whoops, something bad happened
//     at [eval]:5:15
//     at Script.runInThisContext (node:vm:132:18)
//     at Object.runInThisContext (node:vm:309:38)
//     at node:internal/process/execution:77:19
//     at [eval]-wrapper:6:22
//     at evalScript (node:internal/process/execution:76:60)
//     at node:internal/main/eval_string:23:3

const name = 'Will Robinson';
console.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to stderr
```

Example using the `Console` class:

```js
const out = getStreamSomehow();
const err = getStreamSomehow();
const myConsole = new console.Console(out, err);

myConsole.log('hello world');
// Prints: hello world, to out
myConsole.log('hello %s', 'world');
// Prints: hello world, to out
myConsole.error(new Error('Whoops, something bad happened'));
// Prints: [Error: Whoops, something bad happened], to err

const name = 'Will Robinson';
myConsole.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to err
```
@see[source](https://github.com/nodejs/node/blob/v22.x/lib/console.js)console.Console.error(message?: any, ...optionalParams: any[]): void (+1 overload)
Prints to `stderr` with newline. Multiple arguments can be passed, with the
first used as the primary message and all additional used as substitution
values similar to [`printf(3)`](http://man7.org/linux/man-pages/man3/printf.3.html)
(the arguments are all passed to [`util.format()`](https://nodejs.org/docs/latest-v22.x/api/util.html#utilformatformat-args)).

```js
const code = 5;
console.error('error #%d', code);
// Prints: error #5, to stderr
console.error('error', code);
// Prints: error 5, to stderr
```

If formatting elements (e.g. `%d`) are not found in the first string then
[`util.inspect()`](https://nodejs.org/docs/latest-v22.x/api/util.html#utilinspectobject-options) is called on each argument and the
resulting string values are concatenated. See [`util.format()`](https://nodejs.org/docs/latest-v22.x/api/util.html#utilformatformat-args)
for more information.
@sincev0.1.100error('Failed to generate llms.txt:', function (local var) error: unknownerror);
    return new var Response: new (body?: BodyInit | null, init?: ResponseInit) => Response
This Fetch API interface represents the response to a request.

[MDN Reference](https://developer.mozilla.org/docs/Web/API/Response)Response('Error generating llms.txt', { ResponseInit.status?: number | undefinedstatus: 500 });
  }
};

This code accomplishes four key functions:

1. It uses Astro’s getCollection function to grab all published blog posts
2. It sorts them by date with newest first
3. It cleans up each post’s content with helper functions
4. It formats each post with its metadata and content
5. It returns everything as plain text

How to Use It

Using this is simple:

1. Visit alexop.dev/llms.txt in your browser
2. Press Ctrl+A (or Cmd+A on Mac) to select all the text
3. Copy it (Ctrl+C or Cmd+C)
4. Paste it into any LLM with adequate context window (like ChatGPT, Claude, Llama, etc.)
5. Ask questions about your blog content

The LLM now has all your blog posts in its context window. You can ask questions such as:

• “What topics have I written about?”
• “Summarize my post about [topic]”
• “Find code examples in my posts that use [technology]”
• “What have I written about [specific topic]?”

Benefits of This Approach

This approach offers distinct advantages:

• Works with any Astro blog
• Requires a single file to set up
• Makes your content easy to query with any LLM
• Keeps useful metadata with each post
• Formats content in a way LLMs understand well

Conclusion

By adding one straightforward TypeScript file to your Astro blog, you can create a fast way to chat with your own content using any LLM with adequate context window. This makes it easy to:

• Find information in your old posts
• Get summaries of your content
• Find patterns across your writing
• Generate new ideas based on your past content

Give it a try! The setup takes minutes, and it makes interacting with your blog content much faster.