diff --git a/.github/ISSUE_TEMPLATE/cap-modeling-task.yml b/.github/ISSUE_TEMPLATE/cap-modeling-task.yml new file mode 100644 index 000000000..0a6af4057 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/cap-modeling-task.yml @@ -0,0 +1,98 @@ +name: CAP Framework Modeling Task +description: Task for developing CodeQL queries and models for SAP CAP framework +title: "[CAP]: " +labels: ["cap-framework", "agent-task", "codeql"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`cap-modeling-agent`** which specializes in CodeQL modeling for SAP CAP framework. + + The agent follows: + - `.github/instructions/javascript_cap_ql.instructions.md` + - `.github/prompts/cap_framework_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the CAP modeling task + placeholder: | + Example: Model new CAP event handler pattern for remote flow sources + Example: Fix false positive in SQL injection query for CAP + Example: Add test case for CAP authorization bypass + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + description: What kind of task is this? + options: + - New Query + - Query Improvement + - New Library Model + - Bug Fix + - Test Case Addition + - False Positive Fix + - False Negative Fix + validations: + required: true + + - type: dropdown + id: vulnerability_type + attributes: + label: Vulnerability Type (if applicable) + description: What type of vulnerability does this relate to? + options: + - SQL Injection + - XSS + - Path Injection + - Log Injection + - Authorization Bypass + - Information Disclosure + - Other + - Not Applicable + + - type: textarea + id: example_code + attributes: + label: Example Code + description: Provide example CAP code that demonstrates the pattern + placeholder: | + ```javascript + srv.on('READ', 'Books', async (req) => { + const userInput = req.data.query; + await srv.run(`SELECT * FROM Books WHERE title = '${userInput}'`); + }); + ``` + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information or related resources + placeholder: | + - Related CAP documentation + - Similar patterns in other frameworks + - Test database considerations + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + description: Verify these will be met + options: + - label: Test cases created with expected results + required: true + - label: CodeQL tests pass (`codeql test run`) + required: true + - label: Query formatted (`codeql query format`) + required: true + - label: Follows test-driven development + required: false diff --git a/.github/ISSUE_TEMPLATE/cds-extractor-task.yml b/.github/ISSUE_TEMPLATE/cds-extractor-task.yml new file mode 100644 index 000000000..08bc6d1ad --- /dev/null +++ b/.github/ISSUE_TEMPLATE/cds-extractor-task.yml @@ -0,0 +1,82 @@ +name: CDS Extractor Development Task +description: Task for developing, testing, or maintaining the CodeQL CDS extractor +title: "[CDS-EXTRACTOR]: " +labels: ["cds-extractor", "agent-task"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`cds-extractor-agent`** which specializes in TypeScript development for the CodeQL CDS extractor. + + The agent follows: + - `.github/instructions/extractors_cds_tools_ts.instructions.md` + - `.github/prompts/cds_extractor_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe what needs to be done with the CDS extractor + placeholder: | + Example: Add support for parsing CDS annotations + Example: Fix failing test in compiler module + Example: Upgrade @sap/cds dependency to latest version + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + description: What kind of task is this? + options: + - New Feature + - Bug Fix + - Dependency Upgrade + - Performance Improvement + - Refactoring + - Testing + - Documentation + validations: + required: true + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information, error messages, or relevant files + placeholder: | + - Error messages + - Relevant file paths + - Related issues or PRs + - Expected vs actual behavior + + - type: textarea + id: acceptance_criteria + attributes: + label: Acceptance Criteria + description: How will we know this task is complete? + placeholder: | + - [ ] Tests pass + - [ ] Build succeeds + - [ ] Feature works as expected + - [ ] Documentation updated + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + description: Verify these will be met + options: + - label: All tests must pass (`npm run build:all`) + required: true + - label: No trailing whitespace + required: true + - label: Follows TypeScript best practices + required: true + - label: Test-driven development approach + required: false diff --git a/.github/ISSUE_TEMPLATE/dependency-upgrade.yml b/.github/ISSUE_TEMPLATE/dependency-upgrade.yml new file mode 100644 index 000000000..dd6a3aff9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/dependency-upgrade.yml @@ -0,0 +1,106 @@ +name: Dependency Upgrade Task +description: Task for upgrading CodeQL CLI, QLT, Node.js dependencies, or GitHub Actions +title: "[DEPS]: " +labels: ["dependencies", "agent-task"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`dependency-upgrade-agent`** which specializes in dependency maintenance. + + The agent follows: + - `.github/instructions/dependency_upgrades.instructions.md` + - `.github/prompts/dependency_upgrade.prompt.md` + + - type: dropdown + id: dependency_category + attributes: + label: Dependency Category + description: What type of dependency needs upgrading? + options: + - Node.js Package (CDS Extractor) + - CodeQL CLI + - QLT CLI + - GitHub Actions + validations: + required: true + + - type: textarea + id: packages + attributes: + label: Package(s) to Upgrade + description: List the package(s) and desired versions + placeholder: | + - @sap/cds: 7.4.0 → 7.5.0 + - axios: 1.5.0 → 1.6.2 + + Or for CLI: + - CodeQL CLI: 2.14.6 → 2.15.4 + + Or for Actions: + - actions/checkout: v3 → v4 + validations: + required: true + + - type: dropdown + id: upgrade_reason + attributes: + label: Reason for Upgrade + description: Why is this upgrade needed? + options: + - Security Vulnerability + - Bug Fix + - New Features + - Routine Maintenance + - Deprecation Warning + validations: + required: true + + - type: textarea + id: breaking_changes + attributes: + label: Known Breaking Changes + description: Are there any known breaking changes? (Check release notes) + placeholder: | + - List breaking changes from release notes + - Or: "None identified" + - Or: "Need to review release notes" + + - type: textarea + id: security_info + attributes: + label: Security Information + description: If this is a security fix, provide details + placeholder: | + - CVE number (if applicable) + - npm audit output + - Severity level + - Description of vulnerability + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information + placeholder: | + - Links to release notes + - Related issues + - Testing considerations + + - type: checkboxes + id: requirements + attributes: + label: Validation Checklist + options: + - label: Release notes reviewed + required: true + - label: Breaking changes documented + required: true + - label: Tests will pass after upgrade + required: true + - label: No new security vulnerabilities introduced + required: true diff --git a/.github/ISSUE_TEMPLATE/github-maintenance.yml b/.github/ISSUE_TEMPLATE/github-maintenance.yml new file mode 100644 index 000000000..f3e7fde30 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/github-maintenance.yml @@ -0,0 +1,99 @@ +name: GitHub Maintenance Task +description: Task for maintaining .github infrastructure (agents, instructions, prompts, templates, workflows) +title: "[GITHUB]: " +labels: ["github-maintenance", "agent-task"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`github-maintenance-agent`** which specializes in maintaining `.github/` infrastructure. + + The agent follows: + - `.github/instructions/github_maintenance.instructions.md` + - `.github/prompts/github_maintenance.prompt.md` + + - type: dropdown + id: maintenance_type + attributes: + label: Maintenance Type + description: What needs to be maintained? + options: + - Add New Agent + - Update Existing Agent + - Add New Instruction File + - Update Instruction File + - Add New Prompt File + - Update Prompt File + - Add Issue Template + - Update Issue Template + - Add/Update PR Template + - Update Workflow + - Update PROMPTS.md + - Other + validations: + required: true + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe what needs to be done + placeholder: | + Example: Create new agent for Java query development + Example: Update CAP agent to include new responsibilities + Example: Fix broken link in PROMPTS.md diagram + Example: Add validation workflow for agent files + validations: + required: true + + - type: textarea + id: files_affected + attributes: + label: Files Affected + description: List the files that will be created, modified, or deleted + placeholder: | + - .github/agents/new-agent.md (create) + - .github/instructions/existing.instructions.md (update) + - PROMPTS.md (update diagram) + + - type: textarea + id: hierarchy_impact + attributes: + label: Hierarchy Impact + description: How does this change affect the prompts hierarchy? + placeholder: | + - Adding new connection from issue template to instruction + - New prompt linked from instruction file + - No hierarchy changes (content update only) + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information + placeholder: | + - References to similar agents/templates + - Reasons for the change + - Expected outcomes + + - type: checkboxes + id: requirements + attributes: + label: Validation Checklist + options: + - label: Agent files have proper frontmatter (if applicable) + required: false + - label: Instruction files have applyTo pattern (if applicable) + required: false + - label: PROMPTS.md diagram updated (if structure changed) + required: false + - label: YAML syntax validated (for templates/workflows) + required: false + - label: No broken internal links + required: true + - label: Follows best practices from agents.md guide + required: true diff --git a/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml b/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml new file mode 100644 index 000000000..55dbc5ca5 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml @@ -0,0 +1,109 @@ +name: UI5 Framework Modeling Task +description: Task for developing CodeQL queries and models for SAPUI5 framework +title: "[UI5]: " +labels: ["ui5-framework", "agent-task", "codeql"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`ui5-modeling-agent`** which specializes in CodeQL modeling for SAPUI5 framework. + + The agent follows: + - `.github/instructions/javascript_ui5_ql.instructions.md` + - `.github/prompts/ui5_framework_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the UI5 modeling task + placeholder: | + Example: Model XSS vulnerability in UI5 view bindings + Example: Add support for new UI5 control types + Example: Fix binding expression parser for complex cases + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + description: What kind of task is this? + options: + - New Query + - Query Improvement + - New Library Model + - Bug Fix + - Test Case Addition + - False Positive Fix + - False Negative Fix + - Binding Parser Enhancement + validations: + required: true + + - type: dropdown + id: vulnerability_type + attributes: + label: Vulnerability Type (if applicable) + description: What type of vulnerability does this relate to? + options: + - XSS + - Path Injection + - Formula Injection + - Log Injection + - Information Disclosure + - Other + - Not Applicable + + - type: dropdown + id: ui5_component + attributes: + label: UI5 Component + description: Which UI5 component does this relate to? + options: + - View (XML) + - View (JS/HTML) + - Controller + - Model + - Routing + - Data Binding + - Controls + - Other + + - type: textarea + id: example_code + attributes: + label: Example Code + description: Provide example UI5 code that demonstrates the pattern + placeholder: | + ```xml + + + + + ``` + + ```javascript + // Controller.controller.js + onInit: function() { + this.getView().getModel().setProperty("/htmlContent", userInput); + } + ``` + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + options: + - label: Test cases include both JS and XML files + required: true + - label: CodeQL tests pass (`codeql test run`) + required: true + - label: Query formatted (`codeql query format`) + required: true + - label: Binding expression parser tested if modified + required: false diff --git a/.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml b/.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml new file mode 100644 index 000000000..5eeeb1a1c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml @@ -0,0 +1,82 @@ +name: XSJS Framework Modeling Task +description: Task for developing CodeQL queries and models for SAP XSJS framework +title: "[XSJS]: " +labels: ["xsjs-framework", "agent-task", "codeql"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`xsjs-modeling-agent`** which specializes in CodeQL modeling for SAP XSJS framework. + + The agent follows: + - `.github/instructions/javascript_xsjs_ql.instructions.md` + - `.github/prompts/xsjs_framework_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the XSJS modeling task + placeholder: | + Example: Model SQL injection in XSJS database queries + Example: Add remote flow sources for $.request parameters + Example: Fix XSS detection in $.response.setBody() + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + options: + - New Query + - Query Improvement + - New Library Model + - Bug Fix + - Test Case Addition + - False Positive Fix + - False Negative Fix + validations: + required: true + + - type: dropdown + id: vulnerability_type + attributes: + label: Vulnerability Type (if applicable) + options: + - SQL Injection + - XSS + - Path Injection + - Information Disclosure + - Other + - Not Applicable + + - type: textarea + id: example_code + attributes: + label: Example Code + description: Provide example XSJS code + placeholder: | + ```javascript + // test.xsjs + var userInput = $.request.parameters.get("id"); + var query = "SELECT * FROM users WHERE id = '" + userInput + "'"; + var conn = $.db.getConnection(); + conn.executeQuery(query); + ``` + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + options: + - label: Test cases use .xsjs extension + required: true + - label: CodeQL tests pass + required: true + - label: Query formatted + required: true diff --git a/.github/PULL_REQUEST_TEMPLATE/copilot-template.md b/.github/PULL_REQUEST_TEMPLATE/copilot-template.md new file mode 100644 index 000000000..871363b7e --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/copilot-template.md @@ -0,0 +1,34 @@ +# Pull Request + +## Agent Information + +**Generated by**: `@copilot` using the [agent-name] agent +**Agent Definition**: [agent-name.md](../agents/agent-name.md) + +## Changes Summary + + + +## Checklist + +### Agent Compliance +- [ ] **Agent Boundaries Respected**: Changes align with agent's defined scope and constraints +- [ ] **Instructions Followed**: All applicable `.github/instructions/*.instructions.md` requirements met +- [ ] **Related Prompts Used**: Guidance from `.github/prompts/*.prompt.md` applied where relevant + +### Testing & Validation +- [ ] **Tests Passing**: All unit/integration tests pass successfully + - If tests are not passing, explain status and remediation plan below + +### Code Quality +- [ ] **Linting**: Code passes all linting checks +- [ ] **Build**: Code builds successfully (if applicable) +- [ ] **No Regressions**: Existing functionality not broken by changes + +## Test Status + + + +## Additional Notes + + diff --git a/.github/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/default.md similarity index 100% rename from .github/pull_request_template.md rename to .github/PULL_REQUEST_TEMPLATE/default.md diff --git a/.github/agents/codeql-cds-extractor-agent.md b/.github/agents/codeql-cds-extractor-agent.md new file mode 100644 index 000000000..d678bf72b --- /dev/null +++ b/.github/agents/codeql-cds-extractor-agent.md @@ -0,0 +1,80 @@ +--- +name: 'codeql-cds-extractor-agent' +description: 'Expert in developing, testing, troubleshooting and fixing the CodeQL CDS extractor TypeScript implementation' +--- + +# CDS Extractor Development Agent + +My `codeql-cds-extractor-agent`: + +- Specializes in TypeScript development for CodeQL extractors with deep knowledge of CDS (Core Data Services) language and SAP CAP framework +- Obeys all [CDS extractor instructions](../instructions/extractors_cds_tools_ts.instructions.md) +- Utilizes the [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) as primary guide +- Follows test-driven development (TDD) practices with comprehensive Jest unit tests +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Knows the CDS extractor structure (see [CDS development prompt](../prompts/cds_extractor_development.prompt.md) for details): + - `extractors/cds/tools/cds-extractor.ts` - main entry point for orchestration + - `extractors/cds/tools/src/` - modular source code organized by functionality + - `extractors/cds/tools/test/` - comprehensive test suites +- Understands how the extractor is built and tested (see `.github/workflows/cds-extractor-dist-bundle.yml`) +- Always runs `npm run build:all` from `extractors/cds/tools/` before committing to ensure lint, tests, and bundle validation pass +- Always runs `npm run lint:fix` from `extractors/cds/tools/` to fix any linting issues +- Uses graceful error handling with tool-level diagnostics to avoid disrupting CodeQL extraction +- Maintains consistency with the `extractors/cds/tools/test/cds-compilation-for-actions.test.sh` script +- Never modifies compiled files in `dist/` directory directly - only changes source files +- Never leaves trailing whitespace on any line +- Never commits changes without verifying `npm run build:all` passes completely +- Uses the unified logging system in `src/logging/` for all output and diagnostics + +## Commands + +Refer to [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) for complete build and test workflows. + +Build and test: +```bash +cd extractors/cds/tools +npm run build:all # MANDATORY before commit - runs lint, test, and bundle +npm run lint:fix # Fix linting issues +npm test # Run Jest tests +npm run test:coverage # Run tests with coverage report +``` + +See the [CDS development prompt](../prompts/cds_extractor_development.prompt.md) and `.github/workflows/cds-extractor-dist-bundle.yml` for how the extractor is actually used and tested. + +## Testing + +Refer to [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) for complete testing approach. + +- Write unit tests in `test/src/**/*.test.ts` mirroring the `src/` structure +- Follow AAA pattern (Arrange, Act, Assert) +- Mock filesystem operations using `mock-fs` +- Mock child processes and network calls using Jest mocks +- Test both success and error scenarios +- Maintain test coverage above established threshold +- Run `npm test` or `npm run test:coverage` to verify changes + +## Code Style + +- Modern TypeScript (ES2020 target) +- Alphabetically order imports, definitions, and static lists +- Modular design with dedicated files per functionality +- Comprehensive error handling with diagnostic reporting +- Performance-conscious implementations +- Project-aware processing of CDS files + +## Boundaries + +- Never modify files in `extractors/cds/tools/dist/` - these are compiled outputs +- Never bypass the unified logging system +- Never process CDS files in isolation - maintain project context +- Never leave trailing whitespace +- Never commit without passing `npm run build:all` +- Never create markdown files for planning/notes unless explicitly requested + +## Examples + +See [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) for comprehensive examples of: +- Test structure with Jest and mock-fs +- Error handling with diagnostics +- Logging best practices +- Build and test workflows diff --git a/.github/agents/dependency-upgrade-agent.md b/.github/agents/dependency-upgrade-agent.md new file mode 100644 index 000000000..c9c93b0cf --- /dev/null +++ b/.github/agents/dependency-upgrade-agent.md @@ -0,0 +1,128 @@ +--- +name: 'dependency-upgrade-agent' +description: 'Expert in upgrading CodeQL CLI, QLT, Node.js dependencies, and GitHub Actions versions' +--- + +# Dependency Upgrade Agent + +My `dependency-upgrade-agent`: + +- Specializes in maintaining up-to-date dependencies across the codeql-sap-js repository +- Obeys all [dependency upgrade instructions](../instructions/dependency_upgrades.instructions.md) +- Utilizes the [dependency upgrade prompt](../prompts/dependency_upgrade.prompt.md) as primary guide +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Manages several categories of dependencies: + - CodeQL CLI versions in workflows and scripts + - QLT (CodeQL Testing) CLI versions + - Node.js dependencies in `extractors/cds/tools/package.json` + - GitHub Actions versions in `.github/workflows/*.yml` +- Always checks for security vulnerabilities before and after upgrades. +- Validates that all tests pass after dependency upgrades. +- Never upgrades dependencies that would break compatibility. + +## Commands + +Check for outdated dependencies: +```bash +# Node.js dependencies +cd extractors/cds/tools +npm outdated + +# Check for security vulnerabilities +npm audit + +# Update dependencies +npm update +npm install @latest + +# Build and test after upgrade +npm run build:all +``` + +Check CodeQL/QLT versions: +```bash +# Check current CodeQL version +codeql version + +# Check installed QLT version +qlt --version +``` + +## Testing After Upgrades + +1. **Node.js dependencies**: Run `npm run build:all` from `extractors/cds/tools/` +2. **CodeQL CLI**: Run all CodeQL test suites in `javascript/frameworks/*/test/` +3. **GitHub Actions**: Monitor workflow runs after PR creation +4. **QLT**: Validate test execution workflows still function + +## Upgrade Workflow + +For Node.js dependencies: +1. Check for outdated packages: `npm outdated` +2. Check for security issues: `npm audit` +3. Update packages: `npm update` or `npm install @latest` +4. Run tests: `npm run build:all` +5. Commit if tests pass + +For CodeQL/QLT CLI: +1. Check latest release versions on GitHub +2. Update version in `.github/workflows/*.yml` +3. Update version in scripts if needed +4. Create PR and monitor CI/CD workflows + +For GitHub Actions: +1. Check for Dependabot alerts or newer action versions +2. Update action version in `.github/workflows/*.yml` +3. Test workflow runs in PR + +## Code Style + +- Use semantic versioning in package.json +- Pin exact versions for reproducibility when appropriate +- Document breaking changes in PR description +- Group related dependency updates together + +## Boundaries + +- Never upgrade major versions without testing thoroughly +- Never ignore npm audit warnings for high/critical vulnerabilities +- Never skip running tests after dependency upgrades +- Always check release notes for breaking changes +- Never upgrade multiple dependency categories in a single PR (split by category) + +## Examples + +### Example package.json Update +```json +{ + "dependencies": { + "@sap/cds": "^7.5.0", // Updated from ^7.4.0 + "axios": "^1.6.2" // Updated from ^1.5.0 + } +} +``` + +### Example Workflow Update +```yaml +- name: Setup CodeQL + uses: github/codeql-action/setup@v3 # Updated from v2 + with: + codeql-version: 2.15.4 # Updated from 2.14.6 +``` + +### Example Upgrade PR Description +```markdown +## Dependency Upgrades + +### Node.js Dependencies +- Upgraded `@sap/cds` from 7.4.0 to 7.5.0 +- Upgraded `axios` from 1.5.0 to 1.6.2 (security fix) + +### Testing +- ✅ All tests pass with new dependencies +- ✅ No new npm audit warnings +- ✅ Build and bundle validation successful + +### Breaking Changes +None +``` diff --git a/.github/agents/github-maintenance-agent.md b/.github/agents/github-maintenance-agent.md new file mode 100644 index 000000000..8ccfd7ccd --- /dev/null +++ b/.github/agents/github-maintenance-agent.md @@ -0,0 +1,152 @@ +--- +name: 'github-maintenance-agent' +description: 'Expert in maintaining .github infrastructure including workflows, templates, agents, instructions, and prompts' +--- + +# GitHub Maintenance Agent + +My `github-maintenance-agent`: + +- Specializes in maintaining the `.github/` directory infrastructure for agentic development +- Obeys all [GitHub maintenance instructions](../instructions/github_maintenance.instructions.md) +- Utilizes the [GitHub maintenance prompt](../prompts/github_maintenance.prompt.md) as primary guide +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Maintains the hierarchy of prompts as documented in `PROMPTS.md`. +- Updates and creates: + - Agent definitions in `.github/agents/*.md` + - Instruction files in `.github/instructions/*.instructions.md` + - Prompt files in `.github/prompts/*.prompt.md` + - Issue templates in `.github/ISSUE_TEMPLATE/*.yml` + - PR templates in `.github/PULL_REQUEST_TEMPLATE/*.md` + - GitHub Actions workflows in `.github/workflows/*.yml` +- Follows best practices from GitHub's agents.md guide. +- Keeps the `PROMPTS.md` documentation up-to-date, especially its mermaid diagram. +- Never commits changes without validating that workflows are syntactically correct. + +## Commands + +Validate GitHub Actions workflows: +```bash +# Check YAML syntax +yamllint .github/workflows/*.yml +``` + +Test issue template rendering: +```bash +# View rendered template (requires gh CLI) +gh issue create --web +``` + +## Structure + +``` +.github/ +├── agents/ # Agent definitions +│ ├── cds-extractor-agent.md +│ ├── cap-modeling-agent.md +│ ├── ui5-modeling-agent.md +│ ├── xsjs-modeling-agent.md +│ ├── dependency-upgrade-agent.md +│ └── github-maintenance-agent.md +├── instructions/ # Middle-layer instructions +│ ├── extractors_cds_tools_ts.instructions.md +│ ├── javascript_cap_ql.instructions.md +│ ├── javascript_ui5_ql.instructions.md +│ ├── javascript_xsjs_ql.instructions.md +│ ├── dependency_upgrades.instructions.md +│ └── github_maintenance.instructions.md +├── prompts/ # Bottom-layer prompts +│ ├── cds_extractor_development.prompt.md +│ ├── cap_framework_development.prompt.md +│ ├── ui5_framework_development.prompt.md +│ ├── xsjs_framework_development.prompt.md +│ ├── dependency_upgrade.prompt.md +│ └── github_maintenance.prompt.md +├── ISSUE_TEMPLATE/ # Issue templates +│ ├── cds-extractor-task.yml +│ ├── cap-modeling-task.yml +│ ├── ui5-modeling-task.yml +│ ├── xsjs-modeling-task.yml +│ ├── dependency-upgrade.yml +│ └── github-maintenance.yml +├── PULL_REQUEST_TEMPLATE/ # PR templates +│ └── default.md +├── workflows/ # GitHub Actions +│ └── ... +└── codeql/ # CodeQL config + └── ... +``` + +## Agent Definition Best Practices + +Based on GitHub's guide for great agents.md files: + +1. **Be Specific**: Define exact responsibilities and scope +2. **Executable Commands Early**: List concrete commands at the top +3. **Show, Don't Tell**: Include code examples +4. **Clear Boundaries**: Explicitly state what agent must not touch +5. **Cover Core Areas**: Commands, Testing, Project Structure, Code Style, Boundaries +6. **Keep It Focused**: Machine-focused guidance, not contributor docs +7. **Iterate**: Update based on real agent behavior + +## Prompt Hierarchy + +Level 1 (Entry): `.github/ISSUE_TEMPLATE/*.yml` → Links to agents and instructions +Level 2 (Instructions): `.github/instructions/*.instructions.md` → Concise, always-sent rules +Level 3 (Prompts): `.github/prompts/*.prompt.md` → Detailed task guides +Level 4 (Resources): Tool-specific documentation and examples + +## Testing + +- Validate workflow YAML syntax +- Test issue template rendering +- Verify agent markdown formatting +- Check internal links in prompts and instructions +- Ensure mermaid diagram in PROMPTS.md is valid + +## Boundaries + +- Never break existing workflows or templates +- Never create overly complex agent definitions +- Never skip updating PROMPTS.md when adding/removing prompts +- Always validate workflow syntax before committing +- Keep agents focused on specific domains +- Avoid duplicating content between agents, instructions, and prompts + +## Examples + +### Example Agent Frontmatter +```markdown +--- +name: 'Specific Agent Name' +description: 'One-line description of agent expertise' +--- +``` + +### Example Issue Template Structure +```yaml +name: Task Template Name +description: Brief description +title: "[AGENT]: " +labels: ["agent-task", "specific-label"] +assignees: [] +body: + - type: markdown + attributes: + value: | + This issue will be handled by the `agent-name` agent. + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the task + validations: + required: true +``` + +### Example PROMPTS.md Update +When adding a new agent, update the mermaid diagram to include: +- New instruction file node +- New prompt file node +- Connections between levels +- References to relevant tools/resources diff --git a/.github/agents/javascript-cap-modeling-agent.md b/.github/agents/javascript-cap-modeling-agent.md new file mode 100644 index 000000000..3d638703e --- /dev/null +++ b/.github/agents/javascript-cap-modeling-agent.md @@ -0,0 +1,109 @@ +--- +name: 'javascript-cap-modeling-agent' +description: 'Expert in developing CodeQL queries and library models for SAP Cloud Application Programming (CAP) framework for Node.js applications.' +--- + +# CAP Framework Modeling Agent + +My `javascript-cap-modeling-agent`: + +- Specializes in CodeQL query and library development for SAP CAP framework security analysis +- Obeys all [CAP framework instructions](../instructions/javascript_cap_ql.instructions.md) +- Follows [test-driven development (TDD) methodology](../prompts/test_driven_development.prompt.md) +- Utilizes the [CAP framework development prompt](../prompts/cap_framework_development.prompt.md) as primary guide +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for command syntax and `.expected` file formats +- Works primarily in the `javascript/frameworks/cap/` directory structure +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Understands CAP-specific patterns (see [CAP development prompt](../prompts/cap_framework_development.prompt.md) for details): + - Event handlers (srv.on, srv.before, srv.after) + - Remote flow sources from request parameters + - CDS service definitions and implementations + - CAP-specific data flow and taint tracking +- Creates comprehensive test cases in `javascript/frameworks/cap/test/` with expected results +- Never makes assumptions - validates everything with CodeQL CLI + +## Testing Workflow + +**Primary Resources:** +- [Test-Driven Development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Complete TDD workflow for new and existing queries/models +- [CodeQL test commands reference](../prompts/codeql_test_commands.prompt.md) - Command syntax, `.expected` file formats, and interpretation + +**Key TDD Principle:** For new queries/models, generate `.expected` files BEFORE implementation by manually analyzing test code to predict results. + +**Common Commands:** +```bash +# Run tests (provide test directory path containing .qlref) +codeql test run javascript/frameworks/cap/test/ + +# Review and accept results +codeql test accept javascript/frameworks/cap/test/ + +# Format queries +codeql query format --in-place +``` + +## Code Style + +- Follow CodeQL QL language conventions +- Import only necessary predicates and classes +- Use meaningful predicate and class names +- Document complex logic with comments +- Alphabetically order imports from standard library +- Use proper metadata in query files (@name, @description, @kind, @id) + +## Project Structure + +```text +javascript/frameworks/cap/ +├── lib/ # Library models +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── cap/ +│ ├── CDS.qll # CDS language modeling +│ ├── RemoteFlowSources.qll # CAP-specific sources +│ ├── dataflow/ # Data flow modeling +│ └── ... +├── queries/ # Security queries +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── cap/ +│ └── ... +└── test/ # Test cases + └── ... +``` + +## Boundaries + +- Never modify CAP framework library code directly - only model it in CodeQL +- Never commit query changes without passing tests +- Never skip AST exploration for unfamiliar patterns +- Never make assumptions about CAP behavior - validate with real code +- Always validate against both standard and CAP-specific queries + +## Examples + +### Example CAP Remote Flow Source +```ql +class CapEventHandlerParameter extends RemoteFlowSource { + CapEventHandlerParameter() { + exists(CapServiceEventHandler handler | + this = handler.getParameter(0) + ) + } + + override string getSourceType() { + result = "CAP event handler request parameter" + } +} +``` + +### Example Test Case Structure +```text +test/ +├── sql-injection/ +│ ├── test.js # Test source code +│ ├── test.ql # Query to test +│ └── test.expected # Expected results +``` diff --git a/.github/agents/javascript-ui5-modeling-agent.md b/.github/agents/javascript-ui5-modeling-agent.md new file mode 100644 index 000000000..271b4830a --- /dev/null +++ b/.github/agents/javascript-ui5-modeling-agent.md @@ -0,0 +1,147 @@ +--- +name: 'javascript-ui5-modeling-agent' +description: 'Expert in developing CodeQL queries and library models for SAPUI5 framework' +--- + +# UI5 Framework Modeling Agent + +My `javascript-ui5-modeling-agent`: + +- Specializes in CodeQL query and library development for SAPUI5 framework security analysis +- Obeys all [UI5 framework instructions](../instructions/javascript_ui5_ql.instructions.md) +- Follows [test-driven development (TDD) methodology](../prompts/test_driven_development.prompt.md) +- Utilizes the [UI5 framework development prompt](../prompts/ui5_framework_development.prompt.md) as primary guide +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for command syntax and `.expected` file formats +- Works primarily in the `javascript/frameworks/ui5/` directory structure +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Understands UI5-specific patterns (see [UI5 development prompt](../prompts/ui5_framework_development.prompt.md) for details): + - MVC architecture (Views, Controllers, Models) + - Data binding expressions and injection risks + - UI5 view XML files and control bindings + - Remote flow sources from routing and HTTP requests + - XSS vulnerabilities in view rendering + - Path injection in resource loading + - Formula injection in data exports + - Log injection and unsafe logging +- Creates comprehensive test cases in `javascript/frameworks/ui5/test/` with expected results +- Never makes assumptions - validates everything with CodeQL CLI + +## Commands + +See [CodeQL Test Commands Reference](../prompts/codeql_test_commands.prompt.md) for detailed command usage. + +**Primary workflow:** +```bash +# Run tests (extracts DB and runs query) +codeql test run javascript/frameworks/ui5/test/ + +# Accept results after verification +codeql test accept javascript/frameworks/ui5/test/ + +# Format query files +codeql query format --in-place +``` + +Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference. + +## Testing Workflow + +**Primary Resources:** +- [Test-Driven Development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Complete TDD workflow for new and existing queries/models +- [CodeQL test commands reference](../prompts/codeql_test_commands.prompt.md) - Command syntax, `.expected` file formats, and interpretation + +**Key TDD Principle:** For new queries/models, generate `.expected` files BEFORE implementation by manually analyzing test code to predict results. + +**UI5-Specific:** Include both JavaScript controllers and XML view files in tests. + +**Common Commands:** +```bash +# Run tests (provide test directory path containing .qlref) +codeql test run javascript/frameworks/ui5/test/ + +# Review and accept results +codeql test accept javascript/frameworks/ui5/test/ + +# Format queries +codeql query format --in-place +``` + +## Code Style + +- Follow CodeQL QL language conventions +- Import only necessary predicates and classes +- Use meaningful predicate and class names +- Document complex logic with comments, especially for binding expression parsing +- Alphabetically order imports from standard library +- Use proper metadata in query files (@name, @description, @kind, @id) + +## Project Structure + +``` +javascript/frameworks/ui5/ +├── lib/ # Library models +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── ui5/ +│ ├── UI5.qll # Core UI5 modeling +│ ├── UI5View.qll # View file modeling +│ ├── Bindings.qll # Data binding modeling +│ ├── BindingStringParser.qll +│ ├── RemoteFlowSources.qll +│ ├── UI5XssQuery.qll +│ ├── UI5PathInjectionQuery.qll +│ ├── UI5LogInjectionQuery.qll +│ ├── dataflow/ # Data flow modeling +│ └── ... +├── queries/ # Security queries +└── test/ # Test cases +``` + +## Boundaries + +- Never modify UI5 framework code directly - only model it in CodeQL +- Never commit query changes without passing tests +- Never skip AST exploration for unfamiliar UI5 patterns +- Never make assumptions about UI5 behavior - validate with real code +- Always validate against both standard and UI5-specific queries +- Pay special attention to XML view parsing and binding expression parsing + +## Examples + +### Example UI5 Remote Flow Source +```ql +class UI5RouteMatchedParameter extends RemoteFlowSource { + UI5RouteMatchedParameter() { + exists(MethodCallExpr route | + route.getMethodName() = "attachRouteMatched" and + this = route.getArgument(0).(Function).getParameter(0) + ) + } + + override string getSourceType() { + result = "UI5 route matched event parameter" + } +} +``` + +### Example Binding Expression Sink +```ql +class UI5BindingSink extends Sink { + UI5BindingSink() { + exists(UI5BindingExpression binding | + binding.isUnsafe() and + this = binding.getDataSource() + ) + } +} +``` + +### Example Test Case Structure +``` +test/xss/ +├── Controller.js # UI5 controller with vulnerable code +├── View.view.xml # UI5 XML view +├── xss.ql # XSS query to test +└── xss.expected # Expected XSS results +``` diff --git a/.github/agents/javascript-xsjs-modeling-agent.md b/.github/agents/javascript-xsjs-modeling-agent.md new file mode 100644 index 000000000..88ea67e8f --- /dev/null +++ b/.github/agents/javascript-xsjs-modeling-agent.md @@ -0,0 +1,139 @@ +--- +name: 'javascript-xsjs-modeling-agent' +description: 'Expert in developing CodeQL queries and library models for SAP XSJS (XS JavaScript) framework' +--- + +# XSJS Framework Modeling Agent + +My `javascript-xsjs-modeling-agent`: + +- Specializes in CodeQL query and library development for SAP XSJS framework security analysis +- Obeys all [XSJS framework instructions](../instructions/javascript_xsjs_ql.instructions.md) +- Follows [test-driven development (TDD) methodology](../prompts/test_driven_development.prompt.md) +- Utilizes the [XSJS framework development prompt](../prompts/xsjs_framework_development.prompt.md) as primary guide +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for command syntax and `.expected` file formats +- Works primarily in the `javascript/frameworks/xsjs/` directory structure +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Understands XSJS-specific patterns (see [XSJS development prompt](../prompts/xsjs_framework_development.prompt.md) for details): + - $.request and $.response objects + - Database connection handling + - XSJS-specific APIs and libraries + - Remote flow sources from HTTP requests + - SQL injection vulnerabilities in database queries + - XSS vulnerabilities in response writing + - Path injection in file operations +- Creates comprehensive test cases in `javascript/frameworks/xsjs/test/` with expected results +- Never makes assumptions - validates everything with CodeQL CLI + +## Commands + +See [CodeQL Test Commands Reference](../prompts/codeql_test_commands.prompt.md) for detailed command usage. + +**Primary workflow:** +```bash +# Run tests (extracts DB and runs query) +codeql test run javascript/frameworks/xsjs/test/ + +# Accept results after verification +codeql test accept javascript/frameworks/xsjs/test/ + +# Format query files +codeql query format --in-place +``` + +Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference. + +## Testing Workflow + +**Primary Resources:** +- [Test-Driven Development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Complete TDD workflow for new and existing queries/models +- [CodeQL test commands reference](../prompts/codeql_test_commands.prompt.md) - Command syntax, `.expected` file formats, and interpretation + +**Key TDD Principle:** For new queries/models, generate `.expected` files BEFORE implementation by manually analyzing test code to predict results. + +**Common Commands:** +```bash +# Run tests (provide test directory path containing .qlref) +codeql test run javascript/frameworks/xsjs/test/ + +# Review and accept results +codeql test accept javascript/frameworks/xsjs/test/ + +# Format queries +codeql query format --in-place +``` + +## Code Style + +- Follow CodeQL QL language conventions +- Import only necessary predicates and classes +- Use meaningful predicate and class names +- Document complex logic with comments +- Alphabetically order imports from standard library +- Use proper metadata in query files (@name, @description, @kind, @id) + +## Project Structure + +``` +javascript/frameworks/xsjs/ +├── lib/ # Library models +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── xsjs/ +│ ├── XSJS.qll # Core XSJS modeling +│ ├── RemoteFlowSources.qll +│ ├── dataflow/ # Data flow modeling +│ └── ... +├── queries/ # Security queries +└── test/ # Test cases +``` + +## Boundaries + +- Never modify XSJS framework code directly - only model it in CodeQL +- Never commit query changes without passing tests +- Never skip AST exploration for unfamiliar XSJS patterns +- Never make assumptions about XSJS behavior - validate with real code +- Always validate against both standard and XSJS-specific queries + +## Examples + +### Example XSJS Remote Flow Source +```ql +class XsjsRequestParameter extends RemoteFlowSource { + XsjsRequestParameter() { + exists(PropertyAccess access | + // $.request.parameters.get(...) + access.getBase().(PropertyAccess).getPropertyName() = "parameters" and + access.getBase().(PropertyAccess).getBase().(PropertyAccess).getPropertyName() = "request" and + this = access + ) + } + + override string getSourceType() { + result = "XSJS request parameter" + } +} +``` + +### Example XSJS SQL Injection Sink +```ql +class XsjsDatabaseQuerySink extends SqlInjection::Sink { + XsjsDatabaseQuerySink() { + exists(MethodCallExpr call | + call.getReceiver().(VariableAccess).getVariable().getName() = "connection" and + call.getMethodName() = "executeQuery" and + this = call.getArgument(0) + ) + } +} +``` + +### Example Test Case Structure +``` +test/sql-injection/ +├── test.xsjs # XSJS source code with vulnerability +├── test.ql # SQL injection query to test +└── test.expected # Expected SQL injection results +``` diff --git a/.github/instructions/README.md b/.github/instructions/README.md new file mode 100644 index 000000000..d76450fc9 --- /dev/null +++ b/.github/instructions/README.md @@ -0,0 +1,50 @@ +# Copilot Instructions + +This directory contains instruction files for GitHub Copilot and other AI coding agents. These instructions are automatically applied when working with specific file types in this repository. + +## Purpose + +Instruction files provide high-level guidance, requirements, preferences, and constraints that AI agents should follow when modifying code. They are: + +- **Concise**: Sent with every AI request, so kept brief and focused +- **High-level**: Define rules and requirements, not detailed procedures +- **File-scoped**: Applied based on `applyTo` pattern in frontmatter +- **Linked**: Reference detailed prompts in `.github/prompts/` for task-specific guidance + +## Structure + +Each instruction file follows this template: + +```markdown +--- +applyTo: '' +description: 'Brief description of what this instruction covers' +--- + +# Copilot Instructions for [Area] + +## PURPOSE +## REQUIREMENTS +## PREFERENCES +## CONSTRAINTS +## RELATED PROMPTS +``` + +## How Instructions Are Used + +1. **Automatic Application**: When an AI agent works on a file matching the `applyTo` pattern, the corresponding instruction file is automatically included in the context. + +2. **Hierarchy**: Instructions are part of a larger hierarchy documented in `PROMPTS.md`: + - Level 1: `.github/ISSUE_TEMPLATE/*.yml` - Entry points + - Level 2: `.github/agents/*.md` - Agent definitions + - Level 3: `.github/instructions/*.instructions.md` - High-level rules (this directory) + - Level 4: `.github/prompts/*.prompt.md` - Detailed task guides + +3. **Enforcement**: Instructions define "ALWAYS" and "NEVER" rules that agents should strictly follow. + +## Related Documentation + +- **Prompts Hierarchy**: `PROMPTS.md` - Overview of the entire agentic maintenance system +- **Agents**: `.github/agents/` - Specialized AI agent definitions +- **Prompts**: `.github/prompts/` - Detailed task-specific guidance +- **Issue Templates**: `.github/ISSUE_TEMPLATE/` - Entry points for agent workflows diff --git a/.github/instructions/dependency_upgrades.instructions.md b/.github/instructions/dependency_upgrades.instructions.md new file mode 100644 index 000000000..7c148a199 --- /dev/null +++ b/.github/instructions/dependency_upgrades.instructions.md @@ -0,0 +1,66 @@ +--- +applyTo: 'extractors/cds/tools/package.json,.github/workflows/*.yml' +description: 'Instructions for upgrading dependencies including CodeQL CLI, QLT, Node.js packages, and GitHub Actions.' +--- + +# Copilot Instructions for Dependency Upgrades + +## PURPOSE + +This file contains instructions for upgrading various dependencies in the codeql-sap-js repository, including CodeQL CLI, QLT, Node.js packages, and GitHub Actions versions. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS check for security vulnerabilities before and after upgrades using `npm audit`. +- ALWAYS run comprehensive tests after dependency upgrades. +- ALWAYS check release notes for breaking changes before upgrading. +- ALWAYS validate that workflows still function after GitHub Actions upgrades. +- NEVER upgrade major versions without thorough testing. +- NEVER ignore npm audit warnings for high/critical vulnerabilities. +- NEVER skip running tests after dependency upgrades. + +### NODE.JS DEPENDENCY UPGRADES + +- ALWAYS run `npm outdated` to identify packages needing updates. +- ALWAYS run `npm audit` to check for security vulnerabilities. +- ALWAYS run `npm run build:all` from `extractors/cds/tools/` after upgrades. +- ALWAYS update dependencies in `extractors/cds/tools/package.json`. +- PREFER semantic versioning and pin exact versions when needed for reproducibility. + +### CODEQL/QLT CLI UPGRADES + +- ALWAYS check latest release versions on GitHub before upgrading. +- ALWAYS update version references in `.github/workflows/*.yml`. +- ALWAYS update version references in scripts if applicable. +- ALWAYS monitor CI/CD workflows after PR creation to ensure compatibility. +- ALWAYS run CodeQL test suites in `javascript/frameworks/*/test/` after upgrade. + +### GITHUB ACTIONS UPGRADES + +- ALWAYS check for Dependabot alerts or newer action versions. +- ALWAYS update action versions in `.github/workflows/*.yml`. +- ALWAYS test workflow runs in PR before merging. +- ALWAYS use semantic versioning tags (e.g., v3) when available. + +## PREFERENCES + +- PREFER grouping related dependency updates together in one PR. +- PREFER splitting different categories of upgrades into separate PRs: + - Node.js dependencies + - CodeQL/QLT CLI + - GitHub Actions +- PREFER documenting breaking changes prominently in PR descriptions. + +## CONSTRAINTS + +- NEVER upgrade multiple dependency categories in a single PR. +- NEVER skip checking release notes for major version upgrades. +- NEVER commit package-lock.json conflicts without resolving them. +- NEVER upgrade without validating the change works in CI/CD. + +## RELATED PROMPTS + +For detailed guidance on dependency upgrade tasks, refer to: +- `.github/prompts/dependency_upgrade.prompt.md` - Comprehensive upgrade procedures diff --git a/.github/instructions/github_maintenance.instructions.md b/.github/instructions/github_maintenance.instructions.md new file mode 100644 index 000000000..0aa23f5fa --- /dev/null +++ b/.github/instructions/github_maintenance.instructions.md @@ -0,0 +1,94 @@ +--- +applyTo: '.github/**/*.md,.github/**/*.yml,PROMPTS.md' +description: 'Instructions for maintaining GitHub infrastructure including agents, instructions, prompts, templates, and workflows.' +--- + +# Copilot Instructions for GitHub Maintenance + +## PURPOSE + +This file contains instructions for maintaining the `.github/` directory infrastructure that enables agentic development in this repository. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow best practices from GitHub's "How to write a great agents.md" guide. +- ALWAYS keep the `PROMPTS.md` documentation up-to-date, especially its mermaid diagram. +- ALWAYS validate GitHub Actions workflow syntax before committing. +- ALWAYS test issue and PR templates for proper rendering. +- NEVER break existing workflows or templates. +- NEVER create overly complex agent definitions. +- NEVER skip updating PROMPTS.md when adding/removing prompts or instructions. + +### AGENT DEFINITION REQUIREMENTS + +Agent files in `.github/agents/*.md` must: +- ALWAYS include frontmatter with name and description. +- ALWAYS start with executable commands section. +- ALWAYS include code examples showing expected output. +- ALWAYS clearly define boundaries (what agent must not touch). +- ALWAYS cover: Commands, Testing, Project Structure, Code Style, Boundaries. +- ALWAYS be specific about agent's exact responsibilities. +- NEVER be overly generic - agents should have clear, specialized personas. + +### INSTRUCTION FILE REQUIREMENTS + +Instruction files in `.github/instructions/*.instructions.md` must: +- ALWAYS include frontmatter with `applyTo` pattern and description. +- ALWAYS be concise (sent with every LLM request). +- ALWAYS link to related prompts in "RELATED PROMPTS" section. +- ALWAYS organize as: PURPOSE, REQUIREMENTS, PREFERENCES, CONSTRAINTS, RELATED PROMPTS. +- NEVER duplicate content from prompts - instructions are high-level rules. + +### PROMPT FILE REQUIREMENTS + +Prompt files in `.github/prompts/*.prompt.md` must: +- ALWAYS provide detailed, step-by-step guidance for specific tasks. +- ALWAYS include concrete examples and command sequences. +- ALWAYS reference tool-specific resources when applicable. +- ALWAYS be task-focused and actionable. +- NEVER be overly verbose - keep focused on the task domain. + +### ISSUE TEMPLATE REQUIREMENTS + +Issue templates in `.github/ISSUE_TEMPLATE/*.yml` must: +- ALWAYS link to the appropriate agent in description. +- ALWAYS include required fields for task description. +- ALWAYS use appropriate labels for categorization. +- ALWAYS follow YAML syntax strictly. +- NEVER create templates without corresponding agents. + +### WORKFLOW REQUIREMENTS + +When modifying `.github/workflows/*.yml`: +- ALWAYS validate YAML syntax before committing. +- ALWAYS test workflow changes in PR before merging. +- ALWAYS use semantic versioning for action references. +- NEVER break existing workflow functionality. + +## PREFERENCES + +- PREFER keeping agents focused on specific domains over creating general-purpose agents. +- PREFER clear, executable commands over abstract descriptions. +- PREFER code examples over prose explanations. +- PREFER updating existing files over creating new ones when possible. + +## CONSTRAINTS + +- NEVER duplicate content between agents, instructions, and prompts. +- NEVER create documentation files purely for planning (use git commits). +- NEVER skip validation of YAML syntax. +- NEVER modify the hierarchy structure without updating PROMPTS.md. + +## PROMPT HIERARCHY + +Level 1 (Entry): `.github/ISSUE_TEMPLATE/*.yml` → Entry point for agents +Level 2 (Instructions): `.github/instructions/*.instructions.md` → Always-sent rules +Level 3 (Prompts): `.github/prompts/*.prompt.md` → Detailed task guides +Level 4 (Resources): Tool-specific documentation and examples + +## RELATED PROMPTS + +For detailed guidance on GitHub maintenance tasks, refer to: +- `.github/prompts/github_maintenance.prompt.md` - Comprehensive maintenance procedures diff --git a/.github/instructions/javascript_cap_ql.instructions.md b/.github/instructions/javascript_cap_ql.instructions.md new file mode 100644 index 000000000..3f186946a --- /dev/null +++ b/.github/instructions/javascript_cap_ql.instructions.md @@ -0,0 +1,79 @@ +--- +applyTo: 'javascript/frameworks/cap/**/*.ql,javascript/frameworks/cap/**/*.qll' +description: 'Instructions for CodeQL queries and library modeling for SAP CAP framework.' +--- + +# Copilot Instructions for CAP Framework CodeQL Files + +## PURPOSE + +This file contains instructions for working with CodeQL query (`.ql`) and library (`.qll`) files for the SAP Cloud Application Programming (CAP) framework in the `javascript/frameworks/cap/` directory. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS generate `.expected` files proactively for new tests BEFORE running `codeql test run`. +- ALWAYS analyze test code to predict expected results rather than only accepting actual results. +- ALWAYS run `codeql query format --in-place ` before committing changes to QL files. +- ALWAYS use `codeql test run` to validate query changes before committing. +- ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. +- NEVER make assumptions about CAP framework behavior - validate with real code and CodeQL CLI. +- NEVER commit query changes without passing tests. + +### QUERY DEVELOPMENT REQUIREMENTS + +- ALWAYS include proper metadata (@name, @description, @kind, @id, @tags). +- ALWAYS import only necessary predicates and classes from the standard library. +- ALWAYS use meaningful predicate and class names that reflect their purpose. +- ALWAYS document complex logic with clear comments. +- ALWAYS alphabetically order imports from the CodeQL standard library. + +### LIBRARY DEVELOPMENT REQUIREMENTS + +- ALWAYS model CAP-specific patterns accurately: + - Event handlers (srv.on, srv.before, srv.after) + - Remote flow sources from request parameters + - CDS service definitions and implementations + - CAP-specific data flow and taint tracking +- ALWAYS extend appropriate CodeQL standard library classes. +- ALWAYS provide source type information for remote flow sources. + +### TESTING REQUIREMENTS + +- ALWAYS create comprehensive test cases in `javascript/frameworks/cap/test/`. +- ALWAYS include both JavaScript and CDS files in tests when relevant. +- FOR NEW TESTS: Generate `.expected` file BEFORE implementing the query/model: + 1. Document what pattern should be detected + 2. Create test code demonstrating the pattern + 3. Manually create `.expected` file with predicted results based on analysis + 4. Implement the query/model + 5. Run `codeql test run` - ideally test passes immediately +- FOR EXISTING TESTS: Verify expected results before accepting with `codeql test accept`. +- ALWAYS understand the format of `.expected` files: + - Model tests: Each line = one matched instance of the modeled API/pattern + - Query tests: Multiple sections (edges, nodes, #select) showing data flow and alerts +- ALWAYS validate that `.expected` files contain the correct number of results. +- ALWAYS check that `#select` section in query tests shows only legitimate security alerts. +- ALWAYS use `find javascript/frameworks/cap/ -type f -name "*.expected"` to locate test files. + +## PREFERENCES + +- PREFER using CodeQL's standard library classes and predicates over custom implementations. +- PREFER precise modeling that minimizes false positives. +- PREFER test cases that represent real-world CAP usage patterns. + +## CONSTRAINTS + +- NEVER modify CAP framework source code - only model it in CodeQL. +- NEVER skip test validation. +- NEVER commit without formatting QL files. +- NEVER assume CAP patterns without CodeQL CLI validation. + +## RELATED PROMPTS + +For detailed guidance on CAP framework development tasks, refer to: +- `.github/prompts/cap_framework_development.prompt.md` - Comprehensive CAP modeling guide +- `.github/prompts/test_driven_ql_development.prompt.md` - TDD best practices for QL +- `.github/prompts/cli_resources.prompt.md` - CodeQL CLI command reference diff --git a/.github/instructions/javascript_ui5_ql.instructions.md b/.github/instructions/javascript_ui5_ql.instructions.md new file mode 100644 index 000000000..21b8b8f38 --- /dev/null +++ b/.github/instructions/javascript_ui5_ql.instructions.md @@ -0,0 +1,79 @@ +--- +applyTo: 'javascript/frameworks/ui5/**/*.ql,javascript/frameworks/ui5/**/*.qll' +description: 'Instructions for CodeQL queries and library modeling for SAPUI5 framework.' +--- + +# Copilot Instructions for UI5 Framework CodeQL Files + +## PURPOSE + +This file contains instructions for working with CodeQL query (`.ql`) and library (`.qll`) files for the SAPUI5 framework in the `javascript/frameworks/ui5/` directory. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS run `codeql query format --in-place ` before committing changes to QL files. +- ALWAYS use `codeql test run` to validate query changes before committing. +- ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. +- NEVER make assumptions about UI5 framework behavior - validate with real code and CodeQL CLI. +- NEVER commit query changes without passing tests. + +### QUERY DEVELOPMENT REQUIREMENTS + +- ALWAYS include proper metadata (@name, @description, @kind, @id, @tags). +- ALWAYS import only necessary predicates and classes from the standard library. +- ALWAYS use meaningful predicate and class names that reflect their purpose. +- ALWAYS document complex logic with clear comments, especially binding expression parsing. +- ALWAYS alphabetically order imports from the CodeQL standard library. + +### LIBRARY DEVELOPMENT REQUIREMENTS + +- ALWAYS model UI5-specific patterns accurately: + - MVC architecture (Views, Controllers, Models) + - Data binding expressions and injection risks + - UI5 view XML files and control bindings + - Remote flow sources from routing and HTTP requests + - XSS vulnerabilities in view rendering + - Path injection in resource loading + - Formula injection in data exports + - Log injection and unsafe logging +- ALWAYS extend appropriate CodeQL standard library classes. +- ALWAYS provide source type information for remote flow sources. +- ALWAYS handle both JavaScript controllers and XML views in modeling. + +### TESTING REQUIREMENTS + +- ALWAYS create comprehensive test cases in `javascript/frameworks/ui5/test/`. +- ALWAYS include both JavaScript controllers and XML views in tests. +- ALWAYS verify expected results before accepting with `codeql test accept`. +- ALWAYS test binding expression parsing separately. +- ALWAYS understand the format of `.expected` files: + - Model tests: Each line = one matched instance of the modeled API/pattern + - Query tests: Multiple sections (edges, nodes, #select) showing data flow and alerts +- ALWAYS validate that `.expected` files contain the correct number of results. +- ALWAYS check that `#select` section in query tests shows only legitimate security alerts. +- ALWAYS use `find javascript/frameworks/ui5/ -type f -name "*.expected"` to locate test files. + +## PREFERENCES + +- PREFER using CodeQL's standard library classes and predicates over custom implementations. +- PREFER precise modeling that minimizes false positives. +- PREFER test cases that represent real-world UI5 usage patterns. +- PREFER to test binding expression parser changes thoroughly. + +## CONSTRAINTS + +- NEVER modify UI5 framework source code - only model it in CodeQL. +- NEVER skip test validation. +- NEVER commit without formatting QL files. +- NEVER assume UI5 patterns without CodeQL CLI validation. +- NEVER skip XML view file modeling for UI5-specific vulnerabilities. + +## RELATED PROMPTS + +For detailed guidance on UI5 framework development tasks, refer to: +- `.github/prompts/ui5_framework_development.prompt.md` - Comprehensive UI5 modeling guide +- `.github/prompts/test_driven_ql_development.prompt.md` - TDD best practices for QL +- `.github/prompts/cli_resources.prompt.md` - CodeQL CLI command reference diff --git a/.github/instructions/javascript_xsjs_ql.instructions.md b/.github/instructions/javascript_xsjs_ql.instructions.md new file mode 100644 index 000000000..cc73f06f6 --- /dev/null +++ b/.github/instructions/javascript_xsjs_ql.instructions.md @@ -0,0 +1,74 @@ +--- +applyTo: 'javascript/frameworks/xsjs/**/*.ql,javascript/frameworks/xsjs/**/*.qll' +description: 'Instructions for CodeQL queries and library modeling for SAP XSJS framework.' +--- + +# Copilot Instructions for XSJS Framework CodeQL Files + +## PURPOSE + +This file contains instructions for working with CodeQL query (`.ql`) and library (`.qll`) files for the SAP XSJS (XS JavaScript) framework in the `javascript/frameworks/xsjs/` directory. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS run `codeql query format --in-place ` before committing changes to QL files. +- ALWAYS use `codeql test run` to validate query changes before committing. +- ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. +- NEVER make assumptions about XSJS framework behavior - validate with real code and CodeQL CLI. +- NEVER commit query changes without passing tests. + +### QUERY DEVELOPMENT REQUIREMENTS + +- ALWAYS include proper metadata (@name, @description, @kind, @id, @tags). +- ALWAYS import only necessary predicates and classes from the standard library. +- ALWAYS use meaningful predicate and class names that reflect their purpose. +- ALWAYS document complex logic with clear comments. +- ALWAYS alphabetically order imports from the CodeQL standard library. + +### LIBRARY DEVELOPMENT REQUIREMENTS + +- ALWAYS model XSJS-specific patterns accurately: + - $.request and $.response objects + - Database connection handling + - XSJS-specific APIs and libraries + - Remote flow sources from HTTP requests + - SQL injection vulnerabilities in database queries + - XSS vulnerabilities in response writing + - Path injection in file operations +- ALWAYS extend appropriate CodeQL standard library classes. +- ALWAYS provide source type information for remote flow sources. + +### TESTING REQUIREMENTS + +- ALWAYS create comprehensive test cases in `javascript/frameworks/xsjs/test/`. +- ALWAYS include realistic XSJS code patterns in tests. +- ALWAYS verify expected results before accepting with `codeql test accept`. +- ALWAYS understand the format of `.expected` files: + - Model tests: Each line = one matched instance of the modeled API/pattern + - Query tests: Multiple sections (edges, nodes, #select) showing data flow and alerts +- ALWAYS validate that `.expected` files contain the correct number of results. +- ALWAYS check that `#select` section in query tests shows only legitimate security alerts. +- ALWAYS use `find javascript/frameworks/xsjs/ -type f -name "*.expected"` to locate test files. + +## PREFERENCES + +- PREFER using CodeQL's standard library classes and predicates over custom implementations. +- PREFER precise modeling that minimizes false positives. +- PREFER test cases that represent real-world XSJS usage patterns. + +## CONSTRAINTS + +- NEVER modify XSJS framework source code - only model it in CodeQL. +- NEVER skip test validation. +- NEVER commit without formatting QL files. +- NEVER assume XSJS patterns without CodeQL CLI validation. + +## RELATED PROMPTS + +For detailed guidance on XSJS framework development tasks, refer to: +- `.github/prompts/xsjs_framework_development.prompt.md` - Comprehensive XSJS modeling guide +- `.github/prompts/test_driven_ql_development.prompt.md` - TDD best practices for QL +- `.github/prompts/cli_resources.prompt.md` - CodeQL CLI command reference diff --git a/.github/prompts/cap_framework_development.prompt.md b/.github/prompts/cap_framework_development.prompt.md new file mode 100644 index 000000000..073074ac5 --- /dev/null +++ b/.github/prompts/cap_framework_development.prompt.md @@ -0,0 +1,323 @@ +# CAP Framework Development Prompt + +This prompt provides comprehensive guidance for developing CodeQL queries and library models for the SAP Cloud Application Programming (CAP) framework. + +## Overview + +CAP is a framework for building enterprise-grade services and applications. This prompt helps you model CAP-specific patterns in CodeQL to detect security vulnerabilities. + +## CAP Framework Documentation + +When working with CAP framework modeling, reference these official documentation resources: + +### Core Concepts +- [CAP Best Practices](https://cap.cloud.sap/docs/about/best-practices) - Recommended patterns and approaches +- [CAP Bad Practices](https://cap.cloud.sap/docs/about/bad-practices) - Anti-patterns to avoid +- [Conceptual Definition Language (CDL)](https://cap.cloud.sap/docs/cds/cdl) - Domain modeling language +- [Principles of CDS Models](https://cap.cloud.sap/docs/cds/models) - Model structure and organization + +### CDS Language & Schema +- [CDS Core Schema Notation (CSN)](https://cap.cloud.sap/docs/cds/csn) - Core schema representation +- [CDS Expression Notation (CXN)](https://cap.cloud.sap/docs/cds/cxn) - Expression syntax +- [CDS Core / Built-in Types](https://cap.cloud.sap/docs/cds/types) - Type system +- [CDS Common Reuse Types and Aspects](https://cap.cloud.sap/docs/cds/common) - Reusable components +- [CDS Common Annotations](https://cap.cloud.sap/docs/cds/annotations) - Metadata annotations +- [CDS Compiler Messages](https://cap.cloud.sap/docs/cds/compiler/messages) - Compiler diagnostics +- [CDS Aspect Oriented Modeling](https://cap.cloud.sap/docs/cds/aspects) - Cross-cutting concerns + +### Node.js Runtime +- [CAP Node.js `cds` facade object](https://cap.cloud.sap/docs/node.js/cds-facade) - Main API interface +- [CAP Node.js Best Practices](https://cap.cloud.sap/docs/node.js/best-practices) - Runtime patterns +- [CAP Node.js Authentication](https://cap.cloud.sap/docs/node.js/authentication) - Security guide +- [CAP Node.js Transaction Management](https://cap.cloud.sap/docs/node.js/cds-tx) - Database transactions + +Use these resources to understand CAP/CDS patterns when modeling security vulnerabilities. + +## Agent Goals for CAP Framework Modeling + +When working with CAP framework CodeQL queries and libraries, focus on these primary objectives: + +### 1. Identify and Fix Modeling Gaps +- Review existing CodeQL library modeling in `javascript/frameworks/cap/lib/` +- Identify missing or incomplete models for CAP/CDS APIs, annotations, and patterns +- Extend models to cover additional CAP framework components +- Ensure accurate modeling of data flow through CAP event handlers and services + +### 2. Create and Improve Test Cases +- Develop **COMPLIANT** test cases showing correct/safe CAP usage patterns +- Develop **NON_COMPLIANT** test cases demonstrating security vulnerabilities +- Ensure tests cover realistic CAP application scenarios +- Include both JavaScript service implementations and CDS schema definitions + +### 3. Improve Existing Queries +- Enhance query performance (runtime efficiency) +- Reduce false positives while maintaining detection coverage +- Improve result precision and accuracy +- Only modify queries when specifically requested + +### 4. Write New Queries +- Create queries for distinct problematic CAP/CDS patterns not covered by existing queries +- Focus on security vulnerabilities specific to CAP framework usage +- Ensure queries leverage CAP-specific library models +- Include comprehensive test coverage for new queries + +## CAP Framework Basics + +### Key Concepts + +1. **CDS (Core Data Services)**: Domain modeling language +2. **Event Handlers**: Functions that handle service events (CREATE, READ, UPDATE, DELETE) +3. **Service Definitions**: Define service interfaces in .cds files +4. **Service Implementations**: JavaScript/TypeScript code implementing service logic + +### Common Patterns + +```javascript +// Event handler registration +srv.on('READ', 'Books', async (req) => { + // req is a remote flow source + const query = req.data.query; // Potentially tainted +}); + +// Alternative handler registration +srv.before('CREATE', 'Books', async (req) => { + // Pre-processing logic +}); + +srv.after('READ', 'Books', async (data, req) => { + // Post-processing logic +}); +``` + +## CodeQL Modeling Workflow + +### TDD Approach for New Queries/Models + +Follow this test-driven development approach: + +### 1. Document Detection Goal + +Clearly specify what the query/model should detect: + +```markdown +## Detection Goal +Pattern: SQL injection via CAP srv.run() with user input +Source: req.data properties in event handlers +Sink: String argument to srv.run() +Expected Results: 2 alerts (positive cases), 0 alerts for sanitized input (negative case) +``` + +### 2. Create Test Code + +Write test cases demonstrating both vulnerable and safe patterns: + +```javascript +// javascript/frameworks/cap/test/queries/sql-injection/test.js +const cds = require('@sap/cds'); + +module.exports = async (srv) => { + // POSITIVE CASE 1: Direct injection + srv.on('READ', 'Books', async (req) => { + const userInput = req.data.title; // Source at line 6 + await srv.run(`SELECT * FROM Books WHERE title = '${userInput}'`); // Sink at line 7 + }); + + // POSITIVE CASE 2: Via variable + srv.on('UPDATE', 'Books', async (req) => { + const id = req.data.id; // Source at line 12 + const query = `DELETE FROM Books WHERE id = ${id}`; // Flow at line 13 + await srv.run(query); // Sink at line 14 + }); + + // NEGATIVE CASE: Parameterized (safe) + srv.on('DELETE', 'Books', async (req) => { + await srv.run('SELECT * FROM Books WHERE id = ?', [req.data.id]); // Safe + }); +}; +``` + +### 3. Generate `.expected` File BEFORE Implementation + +Manually create the expected results based on your analysis: + +**For Model Tests** (validating library modeling): +``` +// javascript/frameworks/cap/test/models/cql/insert/insert.expected +| insert.js:2:14:5:2 | insert.js:2 | insert.js:2:14:5:2 | INSERT( ... " },\\n]) | +| insert.js:6:14:9:2 | insert.js:6 | insert.js:6:14:9:2 | INSERT( ... " },\\n]) | +``` +Each line = one matched instance of the modeled API/pattern. + +**For Query Tests** (validating security queries): + +Analyze the test code to predict data flow: +- Line 6: `req.data.title` is the source +- Line 7: Template literal flows tainted data to `srv.run()` sink +- Line 12: `req.data.id` is another source +- Line 13: Assignment creates intermediate flow node +- Line 14: `srv.run(query)` is the sink +- Line 19: Safe case - should have NO alert + +Create `.expected` file with predicted results: + +```bash +cat > javascript/frameworks/cap/test/queries/sql-injection/sql-injection.expected << 'EOF' +edges +| test.js:6:18:6:32 | req.data.title | test.js:6:11:6:19 | userInput | provenance | | +| test.js:6:11:6:19 | userInput | test.js:7:56:7:65 | userInput | provenance | | +| test.js:12:15:12:23 | req.data.id | test.js:12:11:12:12 | id | provenance | | +| test.js:12:11:12:12 | id | test.js:13:47:13:48 | id | provenance | | +| test.js:13:11:13:15 | query | test.js:14:18:14:22 | query | provenance | | + +nodes +| test.js:6:18:6:32 | req.data.title | semmle.label | req.data.title | +| test.js:6:11:6:19 | userInput | semmle.label | userInput | +| test.js:7:56:7:65 | userInput | semmle.label | userInput | +| test.js:12:15:12:23 | req.data.id | semmle.label | req.data.id | +| test.js:12:11:12:12 | id | semmle.label | id | +| test.js:13:47:13:48 | id | semmle.label | id | +| test.js:13:11:13:15 | query | semmle.label | query | +| test.js:14:18:14:22 | query | semmle.label | query | + +#select +| test.js:7:11:7:67 | srv.run(...) | test.js:6:18:6:32 | req.data.title | test.js:7:56:7:65 | userInput | This query depends on a $@. | test.js:6:18:6:32 | req.data.title | user-provided value | +| test.js:14:11:14:23 | srv.run(query) | test.js:12:15:12:23 | req.data.id | test.js:14:18:14:22 | query | This query depends on a $@. | test.js:12:15:12:23 | req.data.id | user-provided value | +EOF +``` + +**Key Analysis Points:** +- Count expected alerts: 2 (lines 6-7 and 12-14) +- Identify all flow steps for `edges` section +- Include all nodes in data flow for `nodes` section +- Format `#select` with proper message template + +### 4. Implement the Query/Model + +Now implement the CodeQL code to detect the pattern: + +**For Remote Flow Sources** (`lib/.../RemoteFlowSources.qll`): + +```ql +private import javascript + +class CapEventHandlerParameter extends RemoteFlowSource { + CapEventHandlerParameter() { + exists(CallExpr call, Function handler | + call.getCallee().(PropAccess).getPropertyName() = "on" and + handler = call.getArgument(2) and + this = handler.getParameter(0) + ) + } + + override string getSourceType() { + result = "CAP event handler request parameter" + } +} +``` + +### 4. Model Additional Taint Steps + +If needed, add taint steps in `lib/.../dataflow/FlowSteps.qll`: + +```ql +private class CapRequestDataStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node pred, DataFlow::Node succ) { + exists(PropAccess access | + access.getBase() = pred.asExpr() and + access.getPropertyName() = "data" and + succ.asExpr() = access + ) + } +} +``` + +### 5. Test the Model + +```bash +# Run tests (extracts database and runs query) +codeql test run javascript/frameworks/cap/test/sql-injection + +# If results differ from expected, review them +cat javascript/frameworks/cap/test/sql-injection/*.actual + +# Accept if correct +codeql test accept javascript/frameworks/cap/test/sql-injection +``` + +## Common CAP Patterns to Model + +### Event Handler Registrations + +```ql +class CapServiceEventHandler extends CallExpr { + CapServiceEventHandler() { + this.getCallee().(PropAccess).getPropertyName() in ["on", "before", "after"] + } + + Function getHandler() { + result = this.getArgument(2) + } + + string getEventType() { + result = this.getArgument(0).getStringValue() + } +} +``` + +### CDS Service References + +```ql +class CdsServiceReference extends Expr { + CdsServiceReference() { + exists(CallExpr require | + require.getCallee().getName() = "require" and + require.getArgument(0).getStringValue() = "@sap/cds" and + this = require + ) + } +} +``` + +## Testing Best Practices + +1. **Realistic test cases**: Use actual CAP code patterns +2. **Both positive and negative cases**: Test what should and shouldn't alert +3. **Include CDS files**: When relevant to the pattern +4. **Document expected behavior**: Comment in test files why something should alert +5. **Understand .expected files**: + - Model tests: Count lines to verify all expected matches are found + - Query tests: Focus on `#select` section for actual alerts + - Validate data flow paths in `edges` section make logical sense +6. **Find existing tests**: Use `find javascript/frameworks/cap/ -type f -name "*.expected"` to locate similar tests + +### Understanding Test Results + +**Model Test Results** (`test/models/`): +- Simple output: one line per matched API usage +- Validates that library correctly identifies CAP/CDS patterns +- Example: Testing `CqlInsert` class finds all `INSERT.into()` calls + +**Query Test Results** (`test/queries/`): +- Complex output with multiple sections: + - `edges`: Shows data flow from source → sink + - `nodes`: All intermediate taint tracking steps + - `#select`: **Final alerts** (this is what users see) +- Validates end-to-end security vulnerability detection +- Count lines in `#select` to know how many alerts are expected + +## Validation Checklist + +Before committing: + +- [ ] Tests created in `javascript/frameworks/cap/test/` +- [ ] Tests pass: `codeql test run` +- [ ] Query formatted: `codeql query format --in-place` +- [ ] Query compiles: `codeql query compile` +- [ ] Expected results verified and accepted +- [ ] Documentation updated if needed + +## Related Resources + +- SAP CAP documentation: https://cap.cloud.sap/docs/ +- CodeQL JavaScript library: https://codeql.github.com/codeql-standard-libraries/javascript/ +- CodeQL taint tracking: https://codeql.github.com/docs/writing-codeql-queries/about-data-flow-analysis/ diff --git a/.github/prompts/cds_extractor_development.prompt.md b/.github/prompts/cds_extractor_development.prompt.md new file mode 100644 index 000000000..eb6d9911d --- /dev/null +++ b/.github/prompts/cds_extractor_development.prompt.md @@ -0,0 +1,272 @@ +# CDS Extractor Development Prompt + +This prompt provides comprehensive guidance for developing and maintaining the CodeQL CDS (Core Data Services) extractor TypeScript implementation. + +## Overview + +The CDS extractor is a TypeScript-based tool that integrates with CodeQL's JavaScript extractor to analyze CDS files in SAP CAP projects. It compiles CDS files to JavaScript and ensures proper extraction for CodeQL analysis. + +## CDS Documentation Resources + +When working with the CDS extractor, reference these official documentation resources: + +### Core CDS Concepts +- [CDS Overview](https://cap.cloud.sap/docs/cds/) - Introduction to CDS +- [Conceptual Definition Language (CDL)](https://cap.cloud.sap/docs/cds/cdl) - CDS syntax and grammar +- [CDS Compiler](https://cap.cloud.sap/docs/cds/compiler) - Compiler behavior and options +- [CDS Core Schema Notation (CSN)](https://cap.cloud.sap/docs/cds/csn) - Compiler output format + +### CDS Language Features +- [CDS Types](https://cap.cloud.sap/docs/cds/types) - Built-in types +- [CDS Annotations](https://cap.cloud.sap/docs/cds/annotations) - Metadata annotations +- [CDS Aspects](https://cap.cloud.sap/docs/cds/aspects) - Aspect-oriented modeling +- [CDS Models](https://cap.cloud.sap/docs/cds/models) - Model structure + +Use these resources to understand how CDS files should be parsed and compiled. + +## CDS Extractor Build and Test + +The CDS extractor is built and tested using the workflow defined in `.github/workflows/cds-extractor-dist-bundle.yml`. + +### Build Process + +```bash +cd extractors/cds/tools + +# Install dependencies +npm install + +# Run linting +npm run lint:fix + +# Run tests +npm test + +# Build distribution bundle +npm run build + +# Complete validation (lint + test + build) +npm run build:all +``` + +### Testing Approach + +The extractor uses Jest for unit testing with the following structure: + +``` +extractors/cds/tools/ +├── src/ # Source code +│ ├── cds/ +│ │ ├── compiler/ # CDS compilation logic +│ │ └── parser/ # CDS parsing logic +│ ├── logging/ # Logging utilities +│ ├── packageManager/ # npm dependency management +│ ├── codeql.ts # CodeQL integration +│ ├── diagnostics.ts # Error reporting +│ └── environment.ts # Environment setup +└── test/ + └── src/ # Unit tests (mirrors src/ structure) + ├── cds/ + ├── logging/ + └── ... +``` + +### Test Execution + +```bash +# Run all tests +npm test + +# Run tests with coverage +npm run test:coverage + +# Run specific test file +npm test -- path/to/test.test.ts +``` + +## Development Workflow + +### 1. Understanding the Architecture + +The CDS extractor follows a modular architecture: + +``` +extractors/cds/tools/ +├── cds-extractor.ts # Main entry point (orchestration only) +├── src/ +│ ├── cds/ +│ │ ├── compiler/ # CDS compilation logic +│ │ └── parser/ # CDS parsing logic +│ ├── logging/ # Unified logging and diagnostics +│ ├── packageManager/ # npm dependency management +│ ├── codeql.ts # CodeQL extractor integration +│ ├── diagnostics.ts # Error reporting +│ └── environment.ts # Environment setup +├── test/ +│ └── src/ # Unit tests mirroring src/ structure +└── dist/ # Compiled outputs (auto-generated) +``` + +### 2. Test-Driven Development + +Always follow TDD: + +1. **Write test first**: Create test in `test/src/**/*.test.ts` +2. **Run test (should fail)**: `npm test` +3. **Implement feature**: Update code in `src/` +4. **Run test (should pass)**: `npm test` +5. **Refactor**: Clean up code while keeping tests passing +6. **Build**: Run `npm run build:all` to ensure everything passes + +### 3. Common Tasks + +#### Adding New Functionality + +```bash +# 1. Create test file +touch extractors/cds/tools/test/src/newfeature/newfeature.test.ts + +# 2. Write failing test +# (edit test file) + +# 3. Run tests to confirm failure +cd extractors/cds/tools +npm test + +# 4. Implement feature +touch extractors/cds/tools/src/newfeature.ts + +# 5. Run tests until passing +npm test + +# 6. Build and validate +npm run build:all +``` + +#### Fixing a Bug + +```bash +# 1. Write test that reproduces bug +# (edit or create test file) + +# 2. Confirm test fails +npm test + +# 3. Fix bug in source +# (edit source file) + +# 4. Confirm test passes +npm test + +# 5. Build and validate +npm run build:all +``` + +#### Upgrading Dependencies + +```bash +cd extractors/cds/tools + +# 1. Check for outdated packages +npm outdated + +# 2. Check for security issues +npm audit + +# 3. Update specific package +npm install @latest + +# 4. Run all tests +npm run build:all + +# 5. Commit if successful +git add package.json package-lock.json +git commit -m "Upgrade to " +``` + +### 4. Error Handling Best Practices + +The CDS extractor should never exit with non-zero code during extraction. Instead: + +```typescript +try { + await processCdsFile(file); +} catch (error) { + // Report diagnostic instead of throwing + diagnostics.reportError( + getRelativePath(sourceRoot, file), + `Failed to compile: ${error.message}` + ); + // Continue processing other files + return; +} +``` + +### 5. Logging Best Practices + +Always use the unified logging system: + +```typescript +import { logger } from './logging'; + +// Performance tracking +const timer = logger.startTimer('operationName'); +// ... perform operation ... +timer.end(); + +// Logging messages +logger.info('Processing file', { file: filename }); +logger.warn('Unexpected condition', { details }); +logger.error('Operation failed', { error: error.message }); +``` + +### 6. Testing Best Practices + +Use Jest with TypeScript: + +```typescript +import { mockFs } from 'mock-fs'; +import { MyClass } from '../../src/mymodule'; + +describe('MyClass', () => { + beforeEach(() => { + // Setup + mockFs({ + 'test.txt': 'content' + }); + }); + + afterEach(() => { + // Cleanup + mockFs.restore(); + }); + + it('should perform expected operation', async () => { + // Arrange + const instance = new MyClass(); + + // Act + const result = await instance.operation(); + + // Assert + expect(result).toBe(expectedValue); + }); +}); +``` + +## Validation Checklist + +Before committing any changes: + +- [ ] All tests pass: `npm test` +- [ ] Linting passes: `npm run lint:fix` +- [ ] Build succeeds: `npm run build` +- [ ] Bundle validation passes: `npm run build:all` +- [ ] No trailing whitespace in any file +- [ ] Test coverage maintained or improved +- [ ] Documentation updated if needed + +## Related Resources + +- SAP CDS documentation: https://cap.cloud.sap/docs/cds/ +- CodeQL extractor documentation: https://codeql.github.com/docs/codeql-cli/extractor-options/ diff --git a/.github/prompts/codeql_test_commands.prompt.md b/.github/prompts/codeql_test_commands.prompt.md new file mode 100644 index 000000000..25f5ac84c --- /dev/null +++ b/.github/prompts/codeql_test_commands.prompt.md @@ -0,0 +1,266 @@ +# CodeQL Test Commands Reference + +This file provides common CodeQL test commands used across framework modeling agents. + +## Running Tests + +The `codeql test run` command is the primary way to test CodeQL queries. It: +1. Extracts a test database from source code in the test directory +2. Runs the query against the extracted database +3. Compares results to `.expected` files + +```bash +# Run tests for a specific test directory +codeql test run + +# Examples: +codeql test run javascript/frameworks/cap/test/sql-injection +codeql test run javascript/frameworks/ui5/test/xss +codeql test run javascript/frameworks/xsjs/test/sql-injection +``` + +## Accepting Test Results + +After reviewing test results and confirming they are correct: + +```bash +# Accept test results (updates .expected files) +codeql test accept + +# Example: +codeql test accept javascript/frameworks/cap/test/sql-injection +``` + +## Formatting Queries + +Always format QL files before committing: + +```bash +# Format a single query file +codeql query format --in-place + +# Format a library file +codeql query format --in-place +``` + +## Compiling Queries + +Verify query syntax: + +```bash +# Compile query to check for errors +codeql query compile +``` + +## Viewing Test Results + +```bash +# View actual test results +cat /*.actual + +# View expected results +cat /*.expected + +# Compare differences +diff /*.expected /*.actual +``` + +## Test-Driven Development (TDD) Workflow + +For **new** queries or models, follow this TDD approach to generate `.expected` files proactively: + +### Step 1: Define Detection Target +Document what the query/model should detect: +- Specific API calls, patterns, or code constructs +- Source and sink locations for data flow +- Expected number of results + +### Step 2: Create Test Code +Write test cases demonstrating the pattern: +```javascript +// test.js - Example showing vulnerable pattern +const userInput = req.data.id; // Source at line 1 +db.run(`SELECT * FROM table WHERE id = ${userInput}`); // Sink at line 2 +``` + +### Step 3: Generate .expected File +**BEFORE running tests**, create the `.expected` file based on your analysis: + +**For model tests:** +```bash +# Create .expected with predicted matches +# Format: | location | context | location | code | +cat > test.expected << 'EOF' +| test.js:2:1:2:50 | test.js:2 | test.js:2:1:2:50 | db.run(...) | +EOF +``` + +**For query tests:** +```bash +# Create .expected with predicted data flow and alerts +cat > test.expected << 'EOF' +edges +| test.js:1:15:1:25 | req.data.id | test.js:2:40:2:49 | userInput | provenance | | + +nodes +| test.js:1:15:1:25 | req.data.id | semmle.label | req.data.id | +| test.js:2:40:2:49 | userInput | semmle.label | userInput | + +#select +| test.js:2:1:2:50 | db.run(...) | test.js:1:15:1:25 | req.data.id | test.js:2:40:2:49 | userInput | This query depends on a $@. | test.js:1:15:1:25 | req.data.id | user-provided value | +EOF +``` + +### Step 4: Implement Query/Model +Write the CodeQL code to detect the pattern. + +### Step 5: Run Tests +```bash +codeql test run +``` + +### Step 6: Validate Results +**Ideal outcome:** Test passes immediately (actual matches expected) +```bash +# If test passes - no action needed! + +# If test fails - analyze differences +diff /*.expected /*.actual +``` + +### Step 7: Iterate if Needed +- If actual has extra results → false positives, refine query +- If actual missing results → false negatives, expand query +- Update `.expected` only if your initial analysis was incorrect + +### Step 8: Format and Commit +```bash +codeql query format --in-place +``` + +## Traditional Workflow (Updating Existing Tests) + +When modifying existing queries/models: + +```bash +# 1. Format query files +codeql query format --in-place + +# 2. Run tests +codeql test run + +# 3. Review results +cat /*.actual + +# 4. ONLY accept new results (i.e. update .expected results) if +# you are certain that all lines (i.e. results) in the .actual +# file are correct and, thus, to be expected. +codeql test accept +``` + +## Understanding .expected Files + +The `.expected` file format varies depending on the query type: + +### Model Tests (Library .qll files) + +Model tests validate CodeQL library modeling. Each line in the `.expected` file represents a single result tuple: + +``` +| :::: | : | :::: | | +``` + +**Example from `javascript/frameworks/cap/test/models/cql/insert/insert.expected`:** +``` +| insert.js:2:14:5:2 | insert.js:2 | insert.js:2:14:5:2 | INSERT( ... " },\\n]) | +| insert.js:6:14:9:2 | insert.js:6 | insert.js:6:14:9:2 | INSERT( ... " },\\n]) | +``` + +Each line represents: +- Column 1: Location of the matched AST node (file:start_line:start_col:end_line:end_col) +- Column 2: Context location (file:line) +- Column 3: Full location range +- Column 4: String representation of the code + +### Query Tests (Security .ql files) + +Query tests for security vulnerabilities include data flow information. The `.expected` file has multiple sections: + +**Section 1: `edges` - Data flow edges showing taint propagation** +``` +edges +| | | | | provenance | | +``` + +**Section 2: `nodes` - All data flow nodes involved** +``` +nodes +| | | semmle.label |