Skip to content

Commit 5ca3d1e

Browse files
chore: update llms.txt
1 parent f634140 commit 5ca3d1e

File tree

1 file changed

+142
-13
lines changed

1 file changed

+142
-13
lines changed

llms.txt

Lines changed: 142 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -28,47 +28,92 @@ The following file types are included:
2828
<file>README.md</file>
2929
<metadata>
3030
path: README.md
31-
size: 2743 bytes
31+
size: 5567 bytes
3232
</metadata>
3333

3434
[![CodeQL Advanced](https://github.com/ngmisl/llmstxt/actions/workflows/codeql.yml/badge.svg)](https://github.com/ngmisl/llmstxt/actions/workflows/codeql.yml)
3535
[![Update llms.txt](https://github.com/ngmisl/llmstxt/actions/workflows/update-llms.yml/badge.svg)](https://github.com/ngmisl/llmstxt/actions/workflows/update-llms.yml)
3636

3737
# llmstxt
3838

39-
A Python tool to compress code files into a single, LLM-friendly text file.
39+
A Python tool for compressing and organizing code files into a single, LLM-friendly text file. This tool is designed to help prepare codebases for analysis by Large Language Models by removing unnecessary content while preserving important semantic information.
4040

4141
## Features
4242

43-
- Preserves important comments and docstrings
44-
- Removes unnecessary content
45-
- Structured, LLM-friendly output
46-
- GitHub Actions integration for automatic updates
43+
### Smart Code Compression
44+
45+
- Preserves docstrings and important comments
46+
- Removes redundant whitespace and formatting
47+
- Maintains code structure and readability
48+
- Handles multiple programming languages
49+
50+
### Language Support
51+
52+
- Python (with AST-based compression)
53+
- JavaScript
54+
- Java
55+
- C/C++
56+
- Shell scripts
57+
- HTML/CSS
58+
- Configuration files (JSON, YAML, TOML, INI)
59+
- Markdown
60+
61+
### LLM-Friendly Output
62+
63+
- XML-style semantic markers
64+
- File metadata and type information
65+
- Organized imports section
66+
- Clear file boundaries
67+
- Consistent formatting
68+
69+
### Automation
70+
71+
- GitHub Actions integration
72+
- Automatic updates on code changes
73+
- CI/CD friendly
4774

4875
## Installation
4976

77+
This project uses [uv](https://github.com/astral-sh/uv) for dependency management, but can also be installed directly with pip.
78+
5079
```bash
80+
# Using pip
5181
pip install git+https://github.com/ngmisl/llmstxt.git
82+
83+
# Using uv (recommended for development)
84+
curl -LsSf https://astral.sh/uv/install.sh | sh
85+
uv pip install .
86+
87+
# For development
88+
uv pip install -e ".[dev]"
5289
```
5390

5491
## Usage
5592

56-
### Command Line
93+
### Local Usage
5794

5895
```bash
59-
# Generate llms.txt in current directory
96+
# Generate llms.txt from current directory
6097
python -m llmstxt
6198

6299
# Or import and use in your code
63100
from llmstxt import generate_llms_txt
64101
generate_llms_txt()
65102
```
66103

104+
The script will:
105+
106+
1. Scan the current directory recursively
107+
2. Process files according to .gitignore rules
108+
3. Generate `llms.txt` with compressed content
109+
67110
### GitHub Actions Integration
68111

69-
To automatically update `llms.txt` in your repository:
112+
There are two ways to use this tool with GitHub Actions:
113+
114+
1. **For Your Own Repository**
70115

71-
1. Create `.github/workflows/update-llms.yml` with:
116+
Create `.github/workflows/update-llms.yml` with:
72117

73118
```yaml
74119
name: Update llms.txt
@@ -126,31 +171,115 @@ jobs:
126171
```
127172

128173
The workflow will:
174+
129175
- Run on push to main/master
130176
- Run on pull requests
131177
- Can be triggered manually
132178
- Generate and commit `llms.txt` automatically
133179

180+
2. **For Remote Repositories**
181+
You can trigger the action for any repository using the GitHub API:
182+
183+
```bash
184+
curl -X POST \
185+
-H "Authorization: token $GITHUB_TOKEN" \
186+
-H "Accept: application/vnd.github.v3+json" \
187+
https://api.github.com/repos/ngmisl/llmstxt/dispatches \
188+
-d '{"event_type": "update-llms", "client_payload": {"repository": "https://github.com/user/repo.git"}}'
189+
```
190+
191+
## Output Format
192+
193+
The generated `llms.txt` file follows this structure:
194+
195+
```python
196+
# Project: llmstxt
197+
198+
## Project Structure
199+
This file contains the compressed and processed contents of the project.
200+
201+
### File Types
202+
- .py
203+
- .js
204+
- .java
205+
...
206+
207+
<file>src/main.py</file>
208+
<metadata>
209+
path: src/main.py
210+
type: py
211+
size: 1234 bytes
212+
</metadata>
213+
214+
<imports>
215+
import ast
216+
from typing import Optional
217+
</imports>
218+
219+
<code lang='python'>
220+
def example():
221+
"""Docstring preserved."""
222+
return True
223+
</code>
224+
225+
<file>src/utils.js</file>
226+
<metadata>
227+
path: src/utils.js
228+
type: js
229+
size: 567 bytes
230+
</metadata>
231+
232+
<code lang='javascript'>
233+
function helper() {
234+
return true;
235+
}
236+
</code>
237+
```
238+
239+
## Configuration
240+
241+
The tool can be configured through function parameters:
242+
243+
```python
244+
generate_llms_txt(
245+
output_file="llms.txt", # Output filename
246+
max_file_size=100 * 1024, # Max file size (100KB)
247+
allowed_extensions=( # Supported file types
248+
".py", ".js", ".java",
249+
".c", ".cpp", ".h", ".hpp",
250+
".sh", ".txt", ".md",
251+
".json", ".xml", ".yaml",
252+
".yml", ".toml", ".ini"
253+
)
254+
)
255+
```
256+
134257
## Development
135258

259+
Requirements:
260+
261+
- Python 3.8+
262+
- [uv](https://github.com/astral-sh/uv) for dependency management (recommended)
263+
136264
```bash
137265
# Clone the repository
138266
git clone https://github.com/ngmisl/llmstxt.git
139267
cd llmstxt
140268

141269
# Install development dependencies
142-
pip install -e ".[dev]"
270+
uv pip install -e ".[dev]"
143271

144272
# Run type checking
145273
mypy llmstxt
146274

147-
# Run linting
275+
# Run linting and formatting
148276
ruff check llmstxt
277+
ruff format llmstxt
149278
```
150279

151280
## License
152281

153-
MIT
282+
MIT License - See LICENSE file for details
154283

155284

156285
<file>LICENSE</file>

0 commit comments

Comments
 (0)