GitHub Actions Required Checks for Conditional Jobs

GitHub has a feature called rulesets which allows you to define rulesets to control various behaviors within a repo.

One common use case for rulesets is to define required status checks which must pass before a PR can be merged into the default branch.

For standard workflows, this is pretty straightforward. You choose the GitHub Actions workflow step you want to validate, and select it. For a workflow with a conditional job, or one which uses a matrix strategy to fan out over a variety of configurations, things get more complex.

For example, this repo conditionally runs tests for services based on which changed in the PR using a dynamic matrix (see: advanced-github-actions-matrix). For any given PR, from 0 to N run-tests steps will have executed and we want to validate any that ran passed.

Which step should I choose?! 😱

Before we continue it is also important to highlight the following (somewhat counterintuitive) behavior:

(See: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks).

This GitHub Issue contains lots of useful discussion about this behavior and how to work with/around it and was the inspiration for Option 3 below.

Option 1: Require ALL THE CHECKS!

We could select each of the run-tests steps as required.

As noted, if a check is skipped, it reports its status as "success", so this would work for ensuring all checks that ran passed.

Pros:

• Does not require modifying the workflow

Cons:

• If a new services is added, you must remember to update the status checks
• The UI will have lots of required checks, many of which are skipped

Option 2: Add Downstream Check to Compile the Result

We could also add a job to the workflow which depends on our dynamic matrix but runs even if it fails or is skipped by specifying job.<job_id>.if: always():

  run-tests-status-check:
    runs-on: ubuntu-24.04
    if: always()
    needs: [filter, run-tests]
    steps:
      - name: Run Tests Status Check
        run: |
          svcs='${{ needs.filter.outputs.services }}'
          echo "services=${svcs}"
          echo "run-tests.result=${{ needs.run-tests.result }}"
          
          # If no services changed, treat as success (no tests to run)
          if  -z "$svcs" ; then
            echo "No matching services changed; passing status check."
            exit 0
          fi
          
          # Otherwise, require the matrix job to have succeeded
          if [[ "${{ needs.run-tests.result }}" == "success" ]]; then
            echo "All test matrix jobs passed."
            exit 0
          else
            echo "::error One or more test matrix jobs failed or were cancelled."
            exit 1
          fi

We can then use this single status check as the required check for our workflow:

Pros:

• A single status check keeps PR UI clean
• Continues working with no modifications when adding services

Cons:

• Requires modifying the workflow
• Adds a few seconds to the critical path (even in the success case)

Option 3: Add Downstream Check Which Raises Failures

A third option is to take advantage of the "skipped = success" behavior and add a job which is skipped UNLESS the upstream jobs fail or are cancelled.

  run-tests-failure-alert:
    runs-on: ubuntu-latest
    needs: [filter, run-tests]
    if: ${{ cancelled() || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'failure') }}
    steps:
      - name: Conditionally Fail Required Check
        shell: bash
        run: |
          echo "::error Some required job has failed!"
          exit 1

Successful Run (failure alert step is skipped)

Failed Run (failure alert runs and fails)

Pros:

• A single status check keeps PR UI clean
• Continues working with no modifications when adding services
• Doesn't add time to the critical path in the success case

Cons:

• Requires modifying the workflow
• The if: ${{ cancelled() || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'failure') }} is relatively complex
• Relies on understanding the skipped = success behavior (job naming + comments can help)

Final Thoughts

The fact that option 1 requires remembering to make a manual update as services are added/removed makes it inherently unsafe.

In my opinion, options 2 & 3 are both reasonable approaches and choosing one over the other is a matter of personal preference. I like option 3 slightly more as implemented here because it avoids inline bash and doesn't extend the the critical path runtime.