Skip to content

Conversation

Copy link

Copilot AI commented Nov 20, 2025

Agentic Maintenance Infrastructure - Complete

This PR implements a comprehensive agentic maintenance infrastructure for the codeql-sap-js repository following the codeql-development-template approach and best practices from GitHub's agents.md guide.

✅ Implementation Complete

Infrastructure Created:

  • 6 specialized agents (CDS extractor, CAP/UI5/XSJS modeling, dependency upgrades, GitHub maintenance)
  • 7 instruction files with file-scoped rules via applyTo patterns
  • 6 detailed prompt files with framework documentation links
  • 6 issue templates as workflow entry points
  • Copilot PR template with agent compliance checklist
  • PROMPTS.md documenting 4-level hierarchy with mermaid diagram
  • copilot-setup-steps.yml workflow for automated environment setup

All Feedback Addressed:

  • ✅ Created Copilot PR template with checklist
  • ✅ Converted all string literals to relative markdown links
  • ✅ Added comprehensive framework documentation links (CAP, UI5, XSJS, CDS)
  • ✅ Created shared CodeQL test commands reference
  • ✅ Fixed command duplication across agents
  • ✅ Enhanced agent goals and clarity
  • ✅ Simplified instructions README
  • ✅ Removed ALL PrintAST and codeql test extract references
  • ✅ Fixed PR template relative paths
  • ✅ Created copilot-setup-steps workflow
  • ✅ Fixed PATH configuration for codeql and qlt
  • ✅ Fixed workflow trigger paths to use existing files
  • ✅ Fixed QLT command syntax to use qlt codeql run install
  • ✅ Fixed environment variable retrieval using qlt codeql get home/path

Workflow Now Working:

  • Uses qlt codeql run install to install CodeQL
  • Retrieves paths with qlt codeql get home and qlt codeql get path
  • Adds CodeQL to $GITHUB_PATH for subsequent steps
  • Verifies both qlt and codeql available via which/PATH
  • Installs QL packs for all frameworks
  • Builds CDS extractor with Node.js
  • Complete verification of all tools

🎯 Ready for Use

The agentic maintenance infrastructure is now complete and functional. Agents can be assigned to issues via the issue templates, and the copilot-setup-steps workflow will properly configure the environment for automated tasks.

Original prompt

This section details on the original issue you should resolve

<issue_title>Agentic maintenance of advanced-security/codeql-sap-js repository</issue_title>
<issue_description>
This issue tracks the implementation and validation of agentic maintenance capabilities for this repository, where "maintenance" is intentionally vague for our use case and applies to any development, testing, or other repository-specific task that has been done before and could reasonably be repeated and validated via some complementary set of agentic AI agents, with each "agent" described by a .github/agents/*.md file and backed by the Copilot Coding Agent (CCA) issue-assignment-to-PR workflow.

Purpose

The overall of this "agentic maintenance" approach is to minimize the amount of human time required purely for the maintenance of this repository

Another way to rephrase this "purpose" would be: All good admins are lazy. And we want to be great admins.

Requirements

Requirements to prepare .github/** for agentic maintenance:

  • Setup subdirectories and .github/** contents for Issue templates, PR templates, agents, instructions, and prompts by following the approach used in advanced-security/codeql-development-template/.github/;
  • Implement a "hierarchy of prompts" similar to advanced-security/codeql-development-template/PROMPTS.md, where we want our .github/ISSUE_TEMPLATE/*.yml templates and our .github/agents/*.md agents to be at the top of the hierarchy of prompts and .github/prompts/*.prompt.md prompts to be at the bottom of the hierarchy and .github/instructions/*.instructions.md instructions to be the middle layer that links to the top and bottom layers of the prompts hierarchy.
  • Ensure .github/agents/*.md agents are create using best practices from the how-to-write-a-great-agents-md-lessons-from-over-2500-repsitories guide. The CCA firewall should allow a fetch of that guide, which should form the basis of your understanding for how to implement a complementary set of .github/agents/*.md agents for:
    • CDS extractor development, testing, node dependency upgrades, etc -- this agent must extend/improve/include the existing .github/instructions/extractors_cds_tools_ts.instructions.md instructions;
    • Development of custom CodeQL .ql queries and .qll library modeling for CAP framework;
    • Development of custom CodeQL .ql queries and .qll library modeling for UI5 framework;
    • Development of custom CodeQL .ql queries and .qll library modeling for XSJS framework;
    • Regular upgrades of codeql and qlt CLI dependencies as well as other actions/software versions used in .github/workflows/*.yml actions workflows of this repository;
    • Maintenance of the .github/** implementation of agents, instructions, prompts, issue templates, PR templates, and actions workflows;

Related Issues and PRs

The following closed Issues and Pull Requests (PRs) from this repository should be fetched for context on the types of issues we expect to be able to assign to our agents and the types of changes required to submit and merge a given PR:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits November 20, 2025 18:18
…s, prompts, and issue templates

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement agentic maintenance for codeql-sap-js repository Implement agentic maintenance infrastructure for automated repository workflows Nov 20, 2025
Copilot AI requested a review from data-douser November 20, 2025 18:27
Copilot finished work on behalf of data-douser November 20, 2025 18:27
Copy link
Collaborator

@data-douser data-douser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Create a checklist for yourself to work through all of the different changes that I requested of you via PR comments. There are many comments, and even then you need to think holistically to ensure that comments are systematically addressed.

Copy link
Author

Copilot AI commented Nov 21, 2025

@copilot Create a checklist for yourself to work through all of the different changes that I requested of you via PR comments. There are many comments, and even then you need to think holistically to ...

Created comprehensive checklist:

PR Comment Remediation Checklist

1. Create Copilot PR Template

  • Create .github/PULL_REQUEST_TEMPLATE/copilot-template.md
  • Include agent identification, boundary respect, instruction compliance, test status
  • Update all 6 agent files to reference this template

2. Fix Agent Command Sections

  • Remove duplicate command content across agents
  • Use markdown links to shared command resources
  • Fix codeql test run command with correct arguments
  • Remove unnecessary codeql test extract and codeql query run duplication

3. Improve Agent Markdown Links

  • Convert string literals to relative-path markdown links in all agents
  • Link agents to their corresponding instructions files
  • Link agents to their corresponding prompts files

4. Enhance Framework Documentation Links

  • CAP: Create prompt with CAP documentation links (best practices, CDL, CDS, NodeJS, security)
  • UI5: Create prompt with UI5 API reference and essential docs
  • XSJS: Create prompt with SAP HANA XS JavaScript reference docs
  • CDS: Add CDS-specific documentation links

5. Improve Agent Clarity

  • CAP: Clarify goals (identify gaps, create tests, improve queries, write new queries)
  • CDS: Document how extractor is built/run/tested (reference workflow)
  • UI5: Similar improvements to CAP/CDS
  • XSJS: Similar improvements to CAP/CDS

6. Simplify Instructions README

  • Remove static "Instruction Files" summarization section
  • Keep README minimal and self-documenting

Working through these systematically...

…links, enhance framework docs, fix commands, simplify README

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copilot finished work on behalf of data-douser November 21, 2025 18:01
Copilot AI requested a review from data-douser November 21, 2025 18:01
@data-douser data-douser marked this pull request as ready for review November 21, 2025 18:29
@data-douser data-douser requested a review from Copilot November 21, 2025 18:48
Copilot finished reviewing on behalf of data-douser November 21, 2025 18:51
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a comprehensive agentic maintenance infrastructure for the codeql-sap-js repository, establishing a hierarchical system of agents, instructions, prompts, and templates to enable AI-driven repository maintenance following GitHub's best practices for agents.md files.

Key Changes:

  • Created 6 specialized agents for different development domains (CDS extractor, CAP/UI5/XSJS modeling, dependency upgrades, GitHub maintenance)
  • Established a 4-level prompt hierarchy documented in PROMPTS.md with mermaid visualization
  • Added comprehensive CodeQL test command reference and framework-specific documentation links

Reviewed changes

Copilot reviewed 27 out of 28 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
PROMPTS.md Documents the complete hierarchy of agents, instructions, prompts, and templates with mermaid diagram
.github/prompts/xsjs_framework_development.prompt.md Detailed guidance for XSJS framework CodeQL modeling with official SAP documentation links
.github/prompts/ui5_framework_development.prompt.md Comprehensive UI5 framework development guide with SAPUI5 SDK references
.github/prompts/github_maintenance.prompt.md Instructions for maintaining .github infrastructure and agentic system
.github/prompts/dependency_upgrade.prompt.md Complete workflow for upgrading Node.js, CodeQL CLI, and GitHub Actions dependencies
.github/prompts/codeql_test_commands.prompt.md Shared reference for CodeQL test commands to avoid duplication across agents
.github/prompts/cds_extractor_development.prompt.md Comprehensive CDS extractor TypeScript development guide with CAP documentation
.github/prompts/cap_framework_development.prompt.md Detailed CAP framework modeling guidance with official CAP documentation links
.github/instructions/javascript_xsjs_ql.instructions.md High-level rules for XSJS CodeQL development applied to .ql/.qll files
.github/instructions/javascript_ui5_ql.instructions.md High-level rules for UI5 CodeQL development applied to .ql/.qll files
.github/instructions/javascript_cap_ql.instructions.md High-level rules for CAP CodeQL development applied to .ql/.qll files
.github/instructions/github_maintenance.instructions.md Rules for maintaining .github infrastructure and documentation
.github/instructions/dependency_upgrades.instructions.md Rules for dependency upgrade workflows
.github/instructions/README.md Overview of the instructions system and hierarchy
.github/agents/xsjs-modeling-agent.md Agent definition for XSJS framework security analysis
.github/agents/ui5-modeling-agent.md Agent definition for SAPUI5 framework security analysis
.github/agents/github-maintenance-agent.md Agent for maintaining agentic infrastructure
.github/agents/dependency-upgrade-agent.md Agent for managing dependency upgrades
.github/agents/cds-extractor-agent.md Agent for CDS extractor TypeScript development
.github/agents/cap-modeling-agent.md Agent for CAP framework security analysis
.github/PULL_REQUEST_TEMPLATE/default.md Standard PR template for human contributors
.github/PULL_REQUEST_TEMPLATE/copilot-template.md Specialized PR template for AI-generated PRs with compliance checklist
.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml Issue template for XSJS framework modeling tasks
.github/ISSUE_TEMPLATE/ui5-modeling-task.yml Issue template for UI5 framework modeling tasks
.github/ISSUE_TEMPLATE/github-maintenance.yml Issue template for GitHub infrastructure maintenance
.github/ISSUE_TEMPLATE/dependency-upgrade.yml Issue template for dependency upgrade tasks
.github/ISSUE_TEMPLATE/cds-extractor-task.yml Issue template for CDS extractor development
.github/ISSUE_TEMPLATE/cap-modeling-task.yml Issue template for CAP framework modeling tasks

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@data-douser
Copy link
Collaborator

@copilot apply changes based on the comments in this thread

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copy link
Author

Copilot AI commented Nov 21, 2025

@copilot apply changes based on the comments in this thread

Applied all changes from the bot reviewer comments:

  1. Removed all incorrect codeql test extract references from instruction files (CAP, UI5, XSJS)
  2. Updated to use codeql test run with PrintAST queries for AST exploration
  3. Fixed PR template path from ../.github/agents/ to ../agents/

All changes align with the shared CodeQL test commands reference which states codeql test run handles extraction automatically. Commit: b5eb0bf

…ement

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copilot finished work on behalf of data-douser November 21, 2025 19:00
Copy link
Collaborator

@data-douser data-douser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot This PR is missing a .github/workflow/copilot-setup-steps.yml actions workflow, which is a fundamental component of the agentic maintenance setup. This omission was my fault as I had not explicitly require that workflow to be created until now. The new copilot-setup-steps.yml workflow should look very similar to the following:

name: "Copilot Setup Steps"

on:
  # Allow manual testing through the repository's "Actions" tab
  workflow_dispatch: {}
  # Automatically run the setup steps when an associated workflow is changed.
  push:
    branches:
      - main
    paths:
      - .github/workflows/copilot-setup-steps.yml
      - qlt.conf.json
  pull_request:
    branches:
      - main
    paths:
      - .github/workflows/copilot-setup-steps.yml
      - qlt.conf.json

jobs:
  # The job MUST be called `copilot-setup-steps` or it will not be picked up by Copilot.
  copilot-setup-steps:
    permissions:
      contents: read
    runs-on: ubuntu-latest
    steps:
      - name: Install QLT
        id: install-qlt
        uses: advanced-security/codeql-development-toolkit/.github/actions/install-qlt@main
        with:
          qlt-version: 'latest'
          add-to-path: true

      - name: Install CodeQL
        id: install-codeql
        shell: bash
        run: |
          echo "Installing CodeQL"
          qlt codeql run install
          echo "-----------------------------"
          echo "CodeQL Home: $QLT_CODEQL_HOME"
          echo "CodeQL Binary: $QLT_CODEQL_PATH"

      - name: Verify Versions of Tooling
        shell: bash
        run: |
          echo -e "Checking CodeQL Version:"
          $QLT_CODEQL_PATH --version

          echo -e "Checking QLT Version:"
          echo "QLT Home: ${{ steps.install-qlt.outputs.qlt-home }}"
          qlt version

      - name: Install QL Packs
        shell: bash
        run: |
          qlt query run install-packs

      - name: Setup Node.js for CDS compilation
        uses: actions/setup-node@v6
        with:
          node-version: '18'
          cache: 'npm'
          cache-dependency-path: 'extractors/cds/tools/package-lock.json'

      - name: Verify Node.js and npm tools
        run: |
          echo "Node.js version: $(node --version)"
          echo "npm version: $(npm --version)"
          echo "npx version: $(npx --version)"
          # Verify npx can access @sap/cds-dk without installing globally
          echo "Testing npx access to @sap/cds-dk..."
          npx --yes --package @sap/cds-dk@latest cds --version || echo "CDS will be installed per-project as needed"

      # Compile .cds files to .cds.json files using the dedicated test script
      - name: Compile CAP CDS files
        run: |
          # Use the dedicated CDS compilation script that includes proper version resolution
          # This script follows the same logic as the CDS extractor's resolveCdsVersions function
          ./extractors/cds/tools/workflow/cds-compilation-for-actions.sh

Note that the proposed copilot-setup-steps workflow ^ is essentially a subset of the run-test-suites job (minus the running of tests) defined in the existing .github/workflows/run-codeql-unit-tests-javascript.yml actions workflow.

…ucture

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copy link
Author

Copilot AI commented Nov 21, 2025

@copilot This PR is missing a .github/workflow/copilot-setup-steps.yml actions workflow, which is a fundamental component of the agentic maintenance setup. This omission was my fault as I had not ex...

Created .github/workflows/copilot-setup-steps.yml with:

  • Job named copilot-setup-steps (required for Copilot pickup)
  • QLT and CodeQL installation steps
  • QL pack installation for all frameworks (CAP, UI5, XSJS)
  • Node.js setup and CDS extractor build
  • Verification steps for complete setup
  • Triggers on workflow_dispatch, push, and PR for relevant file changes

Commit: 147ea77

Copilot finished work on behalf of data-douser November 21, 2025 19:40
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copy link
Author

Copilot AI commented Nov 21, 2025

@copilot apply changes based on the comments in this thread

All bot reviewer comments have been addressed in previous commits (b5eb0bf and a255f05). Additionally, removed all PrintAST references per new requirement since this repository doesn't bundle a PrintAST query.

…l command

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…ommands

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copy link
Author

Copilot AI commented Nov 21, 2025

@copilot apply changes based on the comments in this thread

All bot reviewer comments addressed:

  • Removed incorrect codeql test extract references (commits b5eb0bf, a255f05)
  • Fixed PR template path to ../agents/agent-name.md (commit b5eb0bf)
  • Removed all PrintAST references per new requirement (commit a255f05)

Copilot finished work on behalf of data-douser November 21, 2025 20:56
Fixes an error where the `package.json` and `package-lock.json`
files for the CDS extractor were out of sync, which had prevented
running `npm ci` without error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agentic maintenance of advanced-security/codeql-sap-js repository

2 participants