-
Notifications
You must be signed in to change notification settings - Fork 3
Implement agentic maintenance infrastructure for automated repository workflows #260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…s, prompts, and issue templates Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
data-douser
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot Create a checklist for yourself to work through all of the different changes that I requested of you via PR comments. There are many comments, and even then you need to think holistically to ensure that comments are systematically addressed.
Created comprehensive checklist: PR Comment Remediation Checklist1. Create Copilot PR Template
2. Fix Agent Command Sections
3. Improve Agent Markdown Links
4. Enhance Framework Documentation Links
5. Improve Agent Clarity
6. Simplify Instructions README
Working through these systematically... |
…links, enhance framework docs, fix commands, simplify README Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements a comprehensive agentic maintenance infrastructure for the codeql-sap-js repository, establishing a hierarchical system of agents, instructions, prompts, and templates to enable AI-driven repository maintenance following GitHub's best practices for agents.md files.
Key Changes:
- Created 6 specialized agents for different development domains (CDS extractor, CAP/UI5/XSJS modeling, dependency upgrades, GitHub maintenance)
- Established a 4-level prompt hierarchy documented in PROMPTS.md with mermaid visualization
- Added comprehensive CodeQL test command reference and framework-specific documentation links
Reviewed changes
Copilot reviewed 27 out of 28 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| PROMPTS.md | Documents the complete hierarchy of agents, instructions, prompts, and templates with mermaid diagram |
| .github/prompts/xsjs_framework_development.prompt.md | Detailed guidance for XSJS framework CodeQL modeling with official SAP documentation links |
| .github/prompts/ui5_framework_development.prompt.md | Comprehensive UI5 framework development guide with SAPUI5 SDK references |
| .github/prompts/github_maintenance.prompt.md | Instructions for maintaining .github infrastructure and agentic system |
| .github/prompts/dependency_upgrade.prompt.md | Complete workflow for upgrading Node.js, CodeQL CLI, and GitHub Actions dependencies |
| .github/prompts/codeql_test_commands.prompt.md | Shared reference for CodeQL test commands to avoid duplication across agents |
| .github/prompts/cds_extractor_development.prompt.md | Comprehensive CDS extractor TypeScript development guide with CAP documentation |
| .github/prompts/cap_framework_development.prompt.md | Detailed CAP framework modeling guidance with official CAP documentation links |
| .github/instructions/javascript_xsjs_ql.instructions.md | High-level rules for XSJS CodeQL development applied to .ql/.qll files |
| .github/instructions/javascript_ui5_ql.instructions.md | High-level rules for UI5 CodeQL development applied to .ql/.qll files |
| .github/instructions/javascript_cap_ql.instructions.md | High-level rules for CAP CodeQL development applied to .ql/.qll files |
| .github/instructions/github_maintenance.instructions.md | Rules for maintaining .github infrastructure and documentation |
| .github/instructions/dependency_upgrades.instructions.md | Rules for dependency upgrade workflows |
| .github/instructions/README.md | Overview of the instructions system and hierarchy |
| .github/agents/xsjs-modeling-agent.md | Agent definition for XSJS framework security analysis |
| .github/agents/ui5-modeling-agent.md | Agent definition for SAPUI5 framework security analysis |
| .github/agents/github-maintenance-agent.md | Agent for maintaining agentic infrastructure |
| .github/agents/dependency-upgrade-agent.md | Agent for managing dependency upgrades |
| .github/agents/cds-extractor-agent.md | Agent for CDS extractor TypeScript development |
| .github/agents/cap-modeling-agent.md | Agent for CAP framework security analysis |
| .github/PULL_REQUEST_TEMPLATE/default.md | Standard PR template for human contributors |
| .github/PULL_REQUEST_TEMPLATE/copilot-template.md | Specialized PR template for AI-generated PRs with compliance checklist |
| .github/ISSUE_TEMPLATE/xsjs-modeling-task.yml | Issue template for XSJS framework modeling tasks |
| .github/ISSUE_TEMPLATE/ui5-modeling-task.yml | Issue template for UI5 framework modeling tasks |
| .github/ISSUE_TEMPLATE/github-maintenance.yml | Issue template for GitHub infrastructure maintenance |
| .github/ISSUE_TEMPLATE/dependency-upgrade.yml | Issue template for dependency upgrade tasks |
| .github/ISSUE_TEMPLATE/cds-extractor-task.yml | Issue template for CDS extractor development |
| .github/ISSUE_TEMPLATE/cap-modeling-task.yml | Issue template for CAP framework modeling tasks |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@copilot apply changes based on the comments in this thread |
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Applied all changes from the bot reviewer comments:
All changes align with the shared CodeQL test commands reference which states |
…ement Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot This PR is missing a .github/workflow/copilot-setup-steps.yml actions workflow, which is a fundamental component of the agentic maintenance setup. This omission was my fault as I had not explicitly require that workflow to be created until now. The new copilot-setup-steps.yml workflow should look very similar to the following:
name: "Copilot Setup Steps"
on:
# Allow manual testing through the repository's "Actions" tab
workflow_dispatch: {}
# Automatically run the setup steps when an associated workflow is changed.
push:
branches:
- main
paths:
- .github/workflows/copilot-setup-steps.yml
- qlt.conf.json
pull_request:
branches:
- main
paths:
- .github/workflows/copilot-setup-steps.yml
- qlt.conf.json
jobs:
# The job MUST be called `copilot-setup-steps` or it will not be picked up by Copilot.
copilot-setup-steps:
permissions:
contents: read
runs-on: ubuntu-latest
steps:
- name: Install QLT
id: install-qlt
uses: advanced-security/codeql-development-toolkit/.github/actions/install-qlt@main
with:
qlt-version: 'latest'
add-to-path: true
- name: Install CodeQL
id: install-codeql
shell: bash
run: |
echo "Installing CodeQL"
qlt codeql run install
echo "-----------------------------"
echo "CodeQL Home: $QLT_CODEQL_HOME"
echo "CodeQL Binary: $QLT_CODEQL_PATH"
- name: Verify Versions of Tooling
shell: bash
run: |
echo -e "Checking CodeQL Version:"
$QLT_CODEQL_PATH --version
echo -e "Checking QLT Version:"
echo "QLT Home: ${{ steps.install-qlt.outputs.qlt-home }}"
qlt version
- name: Install QL Packs
shell: bash
run: |
qlt query run install-packs
- name: Setup Node.js for CDS compilation
uses: actions/setup-node@v6
with:
node-version: '18'
cache: 'npm'
cache-dependency-path: 'extractors/cds/tools/package-lock.json'
- name: Verify Node.js and npm tools
run: |
echo "Node.js version: $(node --version)"
echo "npm version: $(npm --version)"
echo "npx version: $(npx --version)"
# Verify npx can access @sap/cds-dk without installing globally
echo "Testing npx access to @sap/cds-dk..."
npx --yes --package @sap/cds-dk@latest cds --version || echo "CDS will be installed per-project as needed"
# Compile .cds files to .cds.json files using the dedicated test script
- name: Compile CAP CDS files
run: |
# Use the dedicated CDS compilation script that includes proper version resolution
# This script follows the same logic as the CDS extractor's resolveCdsVersions function
./extractors/cds/tools/workflow/cds-compilation-for-actions.shNote that the proposed copilot-setup-steps workflow ^ is essentially a subset of the run-test-suites job (minus the running of tests) defined in the existing .github/workflows/run-codeql-unit-tests-javascript.yml actions workflow.
…ucture Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Created
Commit: 147ea77 |
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
All bot reviewer comments have been addressed in previous commits (b5eb0bf and a255f05). Additionally, removed all PrintAST references per new requirement since this repository doesn't bundle a PrintAST query. |
…l command Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…ommands Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
All bot reviewer comments addressed: |
Fixes an error where the `package.json` and `package-lock.json` files for the CDS extractor were out of sync, which had prevented running `npm ci` without error.
Agentic Maintenance Infrastructure - Complete
This PR implements a comprehensive agentic maintenance infrastructure for the codeql-sap-js repository following the codeql-development-template approach and best practices from GitHub's agents.md guide.
✅ Implementation Complete
Infrastructure Created:
All Feedback Addressed:
qlt codeql run installqlt codeql get home/pathWorkflow Now Working:
qlt codeql run installto install CodeQLqlt codeql get homeandqlt codeql get path🎯 Ready for Use
The agentic maintenance infrastructure is now complete and functional. Agents can be assigned to issues via the issue templates, and the copilot-setup-steps workflow will properly configure the environment for automated tasks.
Original prompt
This section details on the original issue you should resolve
<issue_title>Agentic maintenance of
advanced-security/codeql-sap-jsrepository</issue_title><issue_description>
This issue tracks the implementation and validation of agentic maintenance capabilities for this repository, where "maintenance" is intentionally vague for our use case and applies to any development, testing, or other repository-specific task that has been done before and could reasonably be repeated and validated via some complementary set of agentic AI agents, with each "agent" described by a
.github/agents/*.mdfile and backed by the Copilot Coding Agent (CCA) issue-assignment-to-PR workflow.Purpose
The overall of this "agentic maintenance" approach is to minimize the amount of human time required purely for the maintenance of this repository
Another way to rephrase this "purpose" would be:
All good admins are lazy.And we want to be great admins.Requirements
Requirements to prepare
.github/**for agentic maintenance:.github/**contents for Issue templates, PR templates, agents, instructions, and prompts by following the approach used in advanced-security/codeql-development-template/.github/;.github/ISSUE_TEMPLATE/*.ymltemplates and our.github/agents/*.mdagents to be at the top of the hierarchy of prompts and.github/prompts/*.prompt.mdprompts to be at the bottom of the hierarchy and.github/instructions/*.instructions.mdinstructions to be the middle layer that links to the top and bottom layers of the prompts hierarchy..github/agents/*.mdagents are create using best practices from the how-to-write-a-great-agents-md-lessons-from-over-2500-repsitories guide. The CCA firewall should allow a fetch of that guide, which should form the basis of your understanding for how to implement a complementary set of.github/agents/*.mdagents for:.github/instructions/extractors_cds_tools_ts.instructions.mdinstructions;.qlqueries and.qlllibrary modeling for CAP framework;.qlqueries and.qlllibrary modeling for UI5 framework;.qlqueries and.qlllibrary modeling for XSJS framework;codeqlandqltCLI dependencies as well as other actions/software versions used in.github/workflows/*.ymlactions workflows of this repository;.github/**implementation of agents, instructions, prompts, issue templates, PR templates, and actions workflows;Related Issues and PRs
The following closed Issues and Pull Requests (PRs) from this repository should be fetched for context on the types of issues we expect to be able to assign to our agents and the types of changes required to submit and merge a given PR:
writeAttributeEscapederroneously included in SinkModel #34advanced-security/codeql-sap-jsrepository #259💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.