Introduction

In our last blog post, we spoke about the Agent Experience (AX) - building tooling to increase the productivity of humans and LLMs alike.

Along those lines, in late 2024, Jeremy Howard, co-founder of Answer.AI, proposed the /llms.txt standard — a format designed to help LLMs effectively process website content.

The /llms.txt standard provides a structured markdown file placed at a website's root directory that offers LLMs concise, well-organized information about the site's content. This approach bypasses the challenges LLMs face when trying to parse complex HTML, JavaScript, and CSS-laden web pages with limited context windows.

# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url)

This format has gained significant traction across the tech industry since its introduction, with companies like Anthropic, Stripe, Mintlify, and many others implementing it to make their documentation more LLM-friendly.

‍

Problems

Database errors have long been a source of frustration for developers. Traditional debugging approaches often involve:

1. Deciphering cryptic error messages
2. Manually cross-referencing table schemas
3. Searching through documentation
4. Trial-and-error query modifications

These approaches are not only time-consuming but also prone to human error. The introduction of LLMs into this workflow has the potential to dramatically reduce debugging time, but only if these models can effectively understand the database context.

While LLMs have shown impressive capabilities in understanding code, they often struggle with database errors for several key reasons:

1. Lack of context: Standard error messages don't include the database schema information that LLMs need to understand relationships between tables
2. Inconsistent formatting: Error logs vary widely between database platforms
3. Missing metadata: Critical information about constraints, data types, and foreign keys is often absent

‍

From llms.txt to llms_database.txt

Inspired by the success of the /llms.txt standard, we've developed llms_database.txt as an extension focused specifically on database debugging contexts.

This file not only encodes a user’s error, but also instructions on the query language (DuckDB), schemas, tables, column definitions, and foreign keys. This file provides contextual metadata LLMs need to provide accurate debugging assistance.

## Prompt
You are an AI assistant helping users debug their DuckDB query errors.

IMPORTANT: When referencing a table, always reference the table with the database AND schema name.
For example, if the table is in the 'main' schema of the 'duckdb' database, you should reference it as
'duckdb.main.table_name'.

## Database Metadata
The database structure is as follows:
{
  "databases": [
    {
      "id": "db_id",
      "name": "duckdb",
      "sizeBytes": 798720,
      "createdAt": "2025-03-30T14:40:31.090678-04:00",
      "lastAccessed": "2025-03-30T14:40:31.090678-04:00",
      "lastSynced": "2025-03-30T14:40:31.090678-04:00",
      "isAttached": true,
      "local": true,
      "schemas": [
        {
          "name": "main",
          "tables": [
            {
              "name": "unicorns",
              "columns": [
                "City",
                "Company",
                "Country",
                "Date Joined",
                "Industry",
                "Select Investors",
                "Valuation ($B)"
              ],
              "column_types": [
                "VARCHAR",
                "VARCHAR",
                "VARCHAR",
                "DATE",
                "VARCHAR",
                "VARCHAR",
                "DOUBLE"
              ],
              "column_null": [
                true,
                true,
                true,
                true,
                true,
                true,
                true
              ]
            }
          ]
        }
      ],
      "region": "us-east-1"
    }
  ]
}    

DuckDB Error: failed to execute prepared query: Binder Error: Referenced column "test" not found in FROM clause!
Candidate bindings: "unicorns.Industry", "unicorns.Date Joined", "unicorns.City"
LINE 2:   test
          ^

Response Format:

## Error Explanation
INSERT EXPLANATION HERE

## Fix Suggestion
INSERT SUGGESTION HERE
    
## Query to Fix
INSERT QUERY HERE

‍

Evaluation Results

We conducted testing to measure the impact of our /llms_database.txt format on debugging performance. The results are compelling:

Note: Evaluation was conducted with developers across various skill levels using real-world database error scenarios

The Future of AI-Ready Content

Our approach aligns with the broader movement toward making technical content more accessible to AI systems.

As Jeremy Howard noted when introducing the /llms.txt standard, the current web wasn't designed with AI in mind. Similarly, database systems weren't built to communicate effectively with LLMs. Our llms_database.txt format bridges this gap, providing the structured, context-rich information that LLMs need to be helpful debugging partners.

As LLMs continue to evolve, we anticipate that formatted database metadata will become an essential component of developer workflows, further reducing the cognitive load associated with database debugging and allowing developers to focus on creating value rather than deciphering cryptic error messages.