update readme

Author: ngmisl

README.md

@@ -3,39 +3,84 @@
 
 # llmstxt
 
-A Python tool to compress code files into a single, LLM-friendly text file.
+A Python tool for compressing and organizing code files into a single, LLM-friendly text file. This tool is designed to help prepare codebases for analysis by Large Language Models by removing unnecessary content while preserving important semantic information.
 
 ## Features
 
-- Preserves important comments and docstrings
-- Removes unnecessary content
-- Structured, LLM-friendly output
-- GitHub Actions integration for automatic updates
+### Smart Code Compression
+
+- Preserves docstrings and important comments
+- Removes redundant whitespace and formatting
+- Maintains code structure and readability
+- Handles multiple programming languages
+
+### Language Support
+
+- Python (with AST-based compression)
+- JavaScript
+- Java
+- C/C++
+- Shell scripts
+- HTML/CSS
+- Configuration files (JSON, YAML, TOML, INI)
+- Markdown
+
+### LLM-Friendly Output
+
+- XML-style semantic markers
+- File metadata and type information
+- Organized imports section
+- Clear file boundaries
+- Consistent formatting
+
+### Automation
+
+- GitHub Actions integration
+- Automatic updates on code changes
+- CI/CD friendly
 
 ## Installation
 
+This project uses [uv](https://github.com/astral-sh/uv) for dependency management, but can also be installed directly with pip.
+
 ```bash
+# Using pip
 pip install git+https://github.com/ngmisl/llmstxt.git
+
+# Using uv (recommended for development)
+curl -LsSf https://astral.sh/uv/install.sh | sh
+uv pip install .
+
+# For development
+uv pip install -e ".[dev]"
 ```
 
 ## Usage
 
-### Command Line
+### Local Usage
 
 ```bash
-# Generate llms.txt in current directory
+# Generate llms.txt from current directory
 python -m llmstxt
 
 # Or import and use in your code
 from llmstxt import generate_llms_txt
 generate_llms_txt()
 ```
 
+The script will:
+
+1. Scan the current directory recursively
+2. Process files according to .gitignore rules
+3. Generate `llms.txt` with compressed content
+
 ### GitHub Actions Integration
 
-To automatically update `llms.txt` in your repository:
+There are two ways to use this tool with GitHub Actions:
 
-1. Create `.github/workflows/update-llms.yml` with:
+1. **For Your Own Repository**
+
+Create `.github/workflows/update-llms.yml` with:
 
 ```yaml
 name: Update llms.txt
@@ -93,28 +138,112 @@ jobs:
 ```
 
 The workflow will:
+
 - Run on push to main/master
 - Run on pull requests
 - Can be triggered manually
 - Generate and commit `llms.txt` automatically
 
+2. **For Remote Repositories**
+   You can trigger the action for any repository using the GitHub API:
+
+```bash
+curl -X POST \
+  -H "Authorization: token $GITHUB_TOKEN" \
+  -H "Accept: application/vnd.github.v3+json" \
+  https://api.github.com/repos/ngmisl/llmstxt/dispatches \
+  -d '{"event_type": "update-llms", "client_payload": {"repository": "https://github.com/user/repo.git"}}'
+```
+
+## Output Format
+
+The generated `llms.txt` file follows this structure:
+
+```python
+# Project: llmstxt
+
+## Project Structure
+This file contains the compressed and processed contents of the project.
+
+### File Types
+- .py
+- .js
+- .java
+...
+
+<file>src/main.py</file>
+<metadata>
+path: src/main.py
+type: py
+size: 1234 bytes
+</metadata>
+
+<imports>
+import ast
+from typing import Optional
+</imports>
+
+<code lang='python'>
+def example():
+    """Docstring preserved."""
+    return True
+</code>
+
+<file>src/utils.js</file>
+<metadata>
+path: src/utils.js
+type: js
+size: 567 bytes
+</metadata>
+
+<code lang='javascript'>
+function helper() {
+  return true;
+}
+</code>
+```
+
+## Configuration
+
+The tool can be configured through function parameters:
+
+```python
+generate_llms_txt(
+    output_file="llms.txt",      # Output filename
+    max_file_size=100 * 1024,    # Max file size (100KB)
+    allowed_extensions=(         # Supported file types
+        ".py", ".js", ".java",
+        ".c", ".cpp", ".h", ".hpp",
+        ".sh", ".txt", ".md",
+        ".json", ".xml", ".yaml",
+        ".yml", ".toml", ".ini"
+    )
+)
+```
+
 ## Development
 
+Requirements:
+
+- Python 3.8+
+- [uv](https://github.com/astral-sh/uv) for dependency management (recommended)
+
 ```bash
 # Clone the repository
 git clone https://github.com/ngmisl/llmstxt.git
 cd llmstxt
 
 # Install development dependencies
-pip install -e ".[dev]"
+uv pip install -e ".[dev]"
 
 # Run type checking
 mypy llmstxt
 
-# Run linting
+# Run linting and formatting
 ruff check llmstxt
+ruff format llmstxt
 ```
 
 ## License
 
-MIT
+MIT License - See LICENSE file for details