Add `llms.txt` generation to documentation build process

[VeckoTheGecko]

Is your feature request related to a problem?

Large language models increasingly rely on website information, but face a critical limitation: context windows are too small to handle most websites in their entirety. Converting complex HTML pages with navigation, ads, and JavaScript into LLM-friendly plain text is both difficult and imprecise.

While websites serve both human readers and LLMs, the latter benefit from more concise, expert-level information gathered in a single, accessible location. This is particularly important for use cases like development environments, where LLMs need quick access to programming documentation and APIs.

• https://llmstxt.org/

llms.txt is a file that has been adopted by many documentation websites (e.g., https://docs.astral.sh/uv/llms.txt ) to make docs more readable to LLM agents.

Its possible to add llms.txt file generation to Sphinx. More info on implementation:

sphinx-doc/sphinx#13268 (comment)

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

I've posted in the Scientific Python Discord (https://discord.com/channels/786703927705862175/1436375597059276841/1436375602960666744 ) adding

Has anyone done this for their project? Taking a quick look at numpy , pandas, scipy and sklearn I haven't seen them do it

Hopefully other maintainers chime in with their experience.

[dcherian]

I think it's worth trying if it's just adding another extension to the build

[lsetiawan]

I think this is an awesome thing to have! I'm personally trying to create an xarray AI agent and will need to generate some sort of llms.txt for reference manually. Cool to see that this feature is available for sphinx now. Hopefully it can be integrated to xarray documentation. Thank you all! 😄