Skip to content

Question: Should Pro package tests use matrix exclusion logic? #2003

@justin808

Description

@justin808

Background

The main test workflows (main.yml, examples.yml) have matrix exclusion logic to skip minimum dependency tests on PRs:

exclude:
  - dependency-level: ${{ github.event_name == 'pull_request' && github.ref \!= 'refs/heads/master' && 'minimum' || '' }}

However, the Pro package test workflows (pro-integration-tests.yml, pro-package-tests.yml) don't have this exclusion logic.

Current Behavior

Pro workflows run when:

if: github.ref == 'refs/heads/master' || github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.run_pro_tests == 'true'

This means:

  • ✅ Pro tests run on master (all matrix combinations)
  • ✅ Pro tests run on workflow_dispatch (all matrix combinations)
  • ✅ Pro tests run on PRs when run_pro_tests == 'true' (all matrix combinations including minimum)

Question

Is this intentional? Should Pro package tests on PRs also skip the minimum dependency matrix?

Option 1: Keep current behavior (all matrix on PRs)

Rationale: Pro package changes are less frequent and warrant full testing even on PRs.

Option 2: Add matrix exclusion like main workflows

Rationale: Consistency across all workflows, faster PR feedback.

jobs:
  test:
    strategy:
      matrix:
        # ... existing matrix
        exclude:
          - ruby-version: ${{ github.event_name == 'pull_request' && github.ref \!= 'refs/heads/master' && '3.2' || '' }}
            node-version: ${{ github.event_name == 'pull_request' && github.ref \!= 'refs/heads/master' && '20' || '' }}
            dependency-level: ${{ github.event_name == 'pull_request' && github.ref \!= 'refs/heads/master' && 'minimum' || '' }}

Affected Files

  • .github/workflows/pro-integration-tests.yml
  • .github/workflows/pro-package-tests.yml

Priority

Low - This is about optimizing test time vs test coverage, not a bug.

Related

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions