feat: added codecov agent trigger example (#52)

jemeza-codegen · web-flow · commit 3081cfb29f0d · 2025-04-16T17:16:28.000-07:00
* added codecov agent example

* added missing files. updated path names

* Automated pre-commit update
diff --git a/README.md b/README.md
@@ -2,50 +2,35 @@
 
 [![Documentation](https://img.shields.io/badge/docs-docs.codegen.com-blue)](https://docs.codegen.com)
 
-This is a collection of examples using [Codegen](https://codegen.com). You can use these examples to learn how to use Codegen and build custom code transformations.
+This is a collection of examples using [Codegen](https://codegen.com). You can use these examples to learn how to use Codegen and integrate the codegen agent into your workflows.
 
 ## Setup
 
 We recommend using [`uv`](https://github.com/astral-sh/uv) with Python 3.13 for the best experience.
 
-To install Codegen, please follow the [official installation guide](https://docs.codegen.com/introduction/installation). Once Codegen is installed, use these steps to run the examples in this repository:
-
-Install the Codegen CLI globally
-```bash
-uv tool install codegen
-```
-
-Initialize Codegen in your project
-```bash
-codegen init
-```
-
-Activate the virtual environment
+Install Codegen
 ```bash
-source .codegen/.venv/bin/activate
+uv pip install codegen
 ```
 
-Your environment is now ready to run example codemods.
-
-### IDE Configuration (Optional)
+In order to use the codegen API you will need to generate a token [here](https://www.codegen.sh/token)
 
-To configure your IDE for optimal use with Codegen, follow our [IDE setup guide](https://docs.codegen.com/introduction/ide-usage#configuring-your-ide-interpreter).
+Your environment is now ready to use the codegen API!
 
 ## Examples
 
 Within the examples folder, each subdirectory contains a self-contained example with:
-- An explanation of the transformation (`README.md`)
-- A Codegen script that performs the transformation (`run.py`)
-- Sample code to transform, if not using a repository (`input_repo/`)
-
-To see a transformation, simply run the `run.py` script within the desired directory.
+- An explanation of the use case (`README.md`)
+- Files that leverage codegen for powerful applications
 
 ## Learn More
 
 - [Documentation](https://docs.codegen.com)
-- [Getting Started Guide](https://docs.codegen.com/introduction/getting-started)
-- [Tutorials](https://docs.codegen.com/tutorials/at-a-glance)
-- [API Reference](https://docs.codegen.com/api-reference)
+- [API](https://docs.codegen.com/introduction/api)
+- [Agent Capabilities](https://docs.codegen.com/introduction/capabilities)
+- [Prompting](https://docs.codegen.com/introduction/prompting)
+- [Community](https://docs.codegen.com/introduction/community)
+
 
 ## Contributing
 
diff --git a/examples/codecov_agent_trigger/README.md b/examples/codecov_agent_trigger/README.md
@@ -0,0 +1,137 @@
+# Codecov Agent Trigger
+
+[![Documentation](https://img.shields.io/badge/docs-docs.codegen.com-blue)](https://docs.codegen.com)
+
+This example demonstrates how to automatically trigger a Codegen agent to fix code coverage issues when a pull request's coverage falls below a specified threshold. The script integrates with GitHub Actions and Codecov to maintain high code quality standards.
+
+## Overview
+
+The codecov agent trigger consists of two main components:
+
+1. **Coverage Report Processor** (`process_coverage_report.py`)
+   - Parses Codecov XML reports
+   - Evaluates coverage against thresholds
+   - Triggers Codegen agent when needed
+
+2. **Agent Prompt Generator** (`generate_codecov_agent_prompt.py`)
+   - Creates contextual prompts for the Codegen agent
+   - Provides necessary information for test generation
+   - Includes setup and test execution instructions
+
+## How It Works
+
+The workflow operates in several steps:
+
+1. **Coverage Analysis**
+   ```python
+   coverage_data = parse_coverage_xml(xml_file)
+   if coverage_data["coverage_percentage"] < threshold:
+       # Trigger agent to fix coverage
+   ```
+   - Parses XML coverage reports
+   - Extracts key metrics (line coverage, branch coverage)
+   - Compares against defined threshold (77%)
+
+2. **Agent Triggering**
+   ```python
+   from codegen import Agent
+
+   new_agent = Agent(token=token, org_id=ORG_ID)
+   second_task = new_agent.run(generate_codecov_agent_prompt(pr_number, repo))
+   ```
+   - Creates new Codegen agent instance
+   - Generates contextual prompt
+   - Initiates automated fix process
+
+3. **GitHub Integration**
+   ```yaml
+   - name: Codecov Agent Trigger
+     env:
+       PR_NUMBER: ${{ github.event.number }}
+       REPO: ${{ github.repository }}
+       TOKEN: ${{ secrets.CODEGEN_AGENT_TOKEN }}
+     run: |
+       python process_coverage_report.py coverage.xml $PR_NUMBER $REPO $TOKEN
+   ```
+   - Runs as part of CI/CD pipeline
+   - Passes PR context to processor
+   - Manages authentication securely
+
+## Setup
+
+1. Install dependencies:
+   ```bash
+   pip install codegen
+   ```
+
+2. Configure GitHub secrets:
+   - `CODECOV_TOKEN`: Your Codecov API token
+   - `CODEGEN_AGENT_TOKEN`: Your Codegen agent token
+        - Get your codegen token [here](https://www.codegen.sh/token)
+
+3. Add to your GitHub Actions workflow:
+   ```yaml
+   - name: Upload coverage reports to Codecov
+     uses: codecov/codecov-action@v4.5.0
+     with:
+       token: ${{ secrets.CODECOV_TOKEN }}
+       file: coverage.xml
+
+   - name: Codecov Agent Trigger
+     run: python process_coverage_report.py coverage.xml $PR_NUMBER $REPO $TOKEN
+   ```
+
+## Usage
+
+The script is typically run automatically as part of your CI/CD pipeline, but you can also run it manually:
+
+```bash
+python process_coverage_report.py <coverage_xml_file> <pr_number> <repo> <token>
+```
+
+Arguments:
+- `coverage_xml_file`: Path to the coverage XML report
+- `pr_number`: GitHub PR number
+- `repo`: GitHub repository name
+- `token`: Codegen agent token
+
+## Output
+
+When coverage is below threshold:
+```
+WARNING: Coverage 75.50% is below threshold of 77%
+Agent will be notified.
+Agent has been notified. URL: https://codegen.com/tasks/123
+```
+
+When coverage meets threshold:
+```
+Coverage is above threshold.
+```
+
+## Coverage Metrics
+
+The script tracks several key metrics:
+- Line coverage
+- Branch coverage
+- Overall coverage percentage
+- Lines covered vs. total lines
+- Branches covered vs. total branches
+
+## Configuration
+
+The default coverage threshold is set to 77%, but you can modify this in `process_coverage_report.py`:
+
+```python
+threshold = 77  # Modify this value to change the coverage threshold
+```
+
+## Learn More
+
+- [Codecov Documentation](https://docs.codecov.com)
+- [Codegen Documentation](https://docs.codegen.com)
+- [GitHub Actions Documentation](https://docs.github.com/en/actions)
+
+## Contributing
+
+Feel free to submit issues and enhancement requests! We welcome contributions to improve the codecov agent trigger functionality. 
diff --git a/examples/codecov_agent_trigger/action.yml b/examples/codecov_agent_trigger/action.yml
@@ -0,0 +1,69 @@
+name: Backend Unit Tests
+
+on:
+  push:
+    branches:
+      - "main"
+  pull_request:
+    branches:
+      - "main"
+  workflow_dispatch:
+
+jobs:
+  unit-tests:
+    # changing the following value will significantly affect github's billing. Be careful and consult with the team before changing it.
+    runs-on: ubuntu-latest-8
+    concurrency:
+      group: ${{ github.workflow }}-${{ github.ref }}
+      cancel-in-progress: ${{ github.event_name != 'push'}}
+
+    steps:
+      - uses: actions/checkout@v4
+      - name: Setup backend
+        id: setup
+        uses: ./.github/actions/setup-backend
+
+      - name: Test with pytest
+        if: ${{ steps.setup.outputs.skip == 'false' }}
+        timeout-minutes: 5
+        run: |
+          cd path/to/your/package
+          ENV=staging \
+          PYTHONPATH=. \
+          uv run pytest \
+            -n auto \
+            -vv \
+            --cov \
+            --cov-report=xml \
+            path/to/your/tests
+
+      - name: Publish Test Report (Unit tests)
+        uses: mikepenz/action-junit-report@v4
+        if: ${{ steps.setup.outputs.skip == 'false' && (success() || failure()) }} # always publish report even if the tests fail
+        continue-on-error: true
+        with:
+          report_paths: "**/build/test-results/test/TEST.xml"
+          detailed_summary: true
+          check_name: "JUnit Test Report - ${{ github.workflow }}"
+          annotate_only: true
+
+      - name: Upload coverage reports to Codecov
+        if: ${{ steps.setup.outputs.skip == 'false' && (success() || failure()) }} # always upload coverage reports even if the tests fail
+        continue-on-error: true
+        uses: codecov/codecov-action@v4.5.0
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          file: path/to/your/coverage.xml
+          disable_search: true
+          name: ${{ github.workflow }}
+
+      - name: Codecov Agent Trigger
+        if: ${{ steps.setup.outputs.skip == 'false' && (success() || failure()) }}
+        continue-on-error: true
+        env:
+          PR_NUMBER: ${{ github.event.number }}
+          REPO: ${{ github.repository }}
+          TOKEN: ${{ secrets.CODEGEN_AGENT_TOKEN }}
+        run: |
+          cd path/to/your/package
+          uv run python process_coverage_report.py coverage.xml $PR_NUMBER $REPO $TOKEN
diff --git a/examples/codecov_agent_trigger/generate_codecov_agent_prompt.py b/examples/codecov_agent_trigger/generate_codecov_agent_prompt.py
@@ -0,0 +1,29 @@
+def generate_codecov_agent_prompt(pr_number, repo_name):
+    return f"""
+# Overview
+Hello Codegen, I just generated a new PR {pr_number} for repo {repo_name} however, it failed the codecove (code coverage) check.
+Please view the PR checks, see the codecov report, and **update the PR** to fix the codecov check by adding any missing tests.
+
+Please make sure to run the tests you add locally to ensure they are working.
+
+DO NOT COMMIT TESTS IF THEY ARE FAILING.
+
+# Use these commands to run the tests:
+
+# Install Python Dependencies
+```
+cd path/to/your/package
+uv venv
+source .venv/bin/activate
+uv pip install .
+```
+
+
+# Run Tests
+```
+chmod +x ./run_test.sh
+run_test.sh
+```
+
+Good luck!
+"""
diff --git a/examples/codecov_agent_trigger/proces_coverage_report.py b/examples/codecov_agent_trigger/proces_coverage_report.py
@@ -0,0 +1,86 @@
+#!/usr/bin/env python
+
+import sys
+import xml.etree.ElementTree as ET
+from typing import Any
+
+from codegen import Agent
+
+from generate_codecov_agent_prompt import generate_codecov_agent_prompt
+
+ORG_ID = 1
+
+
+def parse_coverage_xml(xml_file: str) -> dict[str, Any]:
+    """Parse the coverage.xml file and extract relevant information.
+
+    Args:
+        xml_file: Path to the coverage XML file.
+
+    Returns:
+        Dictionary containing parsed coverage data.
+    """
+    try:
+        tree = ET.parse(xml_file)
+        root = tree.getroot()
+
+        # Extract overall coverage statistics
+        coverage_data = {
+            "version": root.get("version"),
+            "timestamp": root.get("timestamp"),
+            "lines_covered": int(root.get("lines-covered", 0)),
+            "lines_valid": int(root.get("lines-valid", 0)),
+            "line_rate": float(root.get("line-rate", 0)),
+            "branches_covered": int(root.get("branches-covered", 0)),
+            "branches_valid": int(root.get("branches-valid", 0)),
+            "branch_rate": float(root.get("branch-rate", 0)),
+            "complexity": float(root.get("complexity", 0)),
+        }
+
+        # Calculate overall coverage percentage
+        if coverage_data["lines_valid"] > 0:
+            coverage_data["coverage_percentage"] = coverage_data["lines_covered"] / coverage_data["lines_valid"] * 100
+        else:
+            coverage_data["coverage_percentage"] = 0
+
+        return coverage_data
+
+    except Exception as e:
+        print(f"Error parsing coverage XML: {e}")
+        return {}
+
+
+def main():
+    """Main function to process the coverage report."""
+    if len(sys.argv) < 5:
+        print("Usage: python process_coverage_report.py <coverage_xml_file> <pr_number> <repo> <token>")
+        sys.exit(1)
+
+    xml_file = sys.argv[1]
+    pr_number = sys.argv[2]
+    repo = sys.argv[3]
+    token = sys.argv[4]
+
+    coverage_data = parse_coverage_xml(xml_file)
+    coverage_data["pr_number"] = pr_number
+    coverage_data["repo"] = repo
+    coverage_data["token"] = token
+
+    if not coverage_data:
+        print("Failed to parse coverage data")
+        sys.exit(1)
+
+    # Example: Check if coverage meets a threshold
+    threshold = 77  # 77% coverage threshold
+    if coverage_data["coverage_percentage"] < threshold:
+        print(f"\nWARNING: Coverage {coverage_data['coverage_percentage']:.2f}% is below threshold of {threshold}%")
+        print("Agent will be notified.")
+        new_agent = Agent(token=token, org_id=ORG_ID)
+        second_task = new_agent.run(generate_codecov_agent_prompt(pr_number, repo))
+        print(f"Agent has been notified. URL: {second_task.web_url}")
+    else:
+        print("Coverage is above threshold.")
+
+
+if __name__ == "__main__":
+    main()
diff --git a/examples/codecov_agent_trigger/run_test.sh b/examples/codecov_agent_trigger/run_test.sh
@@ -0,0 +1,10 @@
+#! /bin/bash
+
+# Set environment variables
+export ENV_VAR_1=1
+export ENV_VAR_2=2
+export ENV_VAR_3=3
+
+# Run the tests
+cd path/to/your/package
+pytest -v --cov=./path/to/your/tests --cov-report=xml:coverage.xml "$@"
