Repository navigation
perf(ci): skip Check Tests at the job level when the PR has a diff base - #17553
kyle-sexton wants to merge 2 commits into
Conversation
Check Tests started a runner on almost every pull request, checked out (9-18 s), then exited in shell because a diff base existed: about 369 jobs a day of checkout-and-exit. With a base, check-plugins already scans the changed test files. Select Tests now publishes run_check_tests, true only when the diff base is empty (an unset base also renders true, so the scan runs). Check Tests carries it as an ordered skip in its job condition, and the shell branch is gone. The skip passes CI Status through the existing ordered-skip step (id lanes): it turns a skipped check-tests into success only when Select Tests succeeded and run_check_tests is exactly 'false'; any other skip stays skipped and the aggregate (treat-skipped-as: fail) reds it. The docs-only gate pins the new row and the job condition, and ci-ordered-skips.test.sh covers both directions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeBVgdbwavfuWZGmKnMhWX
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
claude-security-review reviewed Security review: PR #17553, skip Check Tests at the job level when a diff base existsNo security issues found. This PR changes a merge hold: the ci-status ordered-skip step in
Spend: this reduces CI runner usage (about 369 fewer jobs a day, by the PR's estimate). It is not a cost hold. critical-findings: 0 |
|
claude-review reviewed Code review: PR #17553, perf(ci): skip Check Tests at the job level when the PR has a diff base I found nothing to block or flag. I read all 6 changed files in full. What I checked:
Loosened check: not a loosening. The
No input that failed before now passes. Loosened-check verdict: fail (default: the prompt has no Unverified: the pass counts in the PR body (21, 91 and 24 tests, and a clean actionlint run) are the author's claims. I ran nothing to confirm them. Spend: none. The change only removes runner jobs: about 369 a day of Verifier dropped 0 of 0 candidates. |
|
The pr-verify verifier failed head What it wrote (33 lines)This pull request leaves the merge queue, and ci-status holds it out while it carries |
No related issue: GitHub Optimization loop cut (Check Tests runs a runner only when it has work).
Summary
Check Testsstarted a runner on every PR, checked out the tree, and then exited at once whenever a diff base existed, becausecheck-pluginsalready scans the changed test files. That was a 9-18 s checkout with 0 s of work (runs 44614871783, 44622815071), about 369 jobs and 132 runner-minutes a day. The branch lived in the shell because a skipped job used to turnCI Statusred. The ordered-skip check now handles that.Fix
Select Testspublishesrun_check_tests: ${{ steps.base.outputs.ref == '' }}. An unset base renders true, so the scan runs.Check Tests' jobif:adds&& needs.select-tests.outputs.run_check_tests == 'true'. The shell branch andDIFF_BASEare gone.ci-status's ordered-skip step (Check each skipped lane against scope) addscheck-teststo its lane pattern and maps the lane to its row withgsub("-"; "_"), socheck-testsreadsrun_check_tests. It turnsskippedintosuccessonly whenselect-testssucceeded and that row is exactly"false". Any other skip staysskipped, andtreat-skipped-as: failfails it.scripts/check-docs-only-gate.shlists the new row and allows this job condition, since it rejects any output or jobif:it does not list.Verification
bash scripts/ci-ordered-skips.test.sh: 21 pass, 0 fail. It runs the workflow's own step body. Cases include: check-tests skipped beside a diff base passes; skipped with no diff base stays skipped; an empty row stays skipped; a run that failed keeps its failure; other lanes do not borrow the row.bash scripts/check-docs-only-gate.test.sh: 91 pass, 0 fail, including a case that rejects a!= ''row.bash scripts/check-cant-fail-scan.test.sh: 24 pass, 0 fail.actionlint .github/workflows/pr-require-checks.yml: clean.Expected after merge: Check Tests shows as skipped on PRs with a diff base,
CI Statusstays green, and about 369 jobs/day disappear. Merge latency is unchanged, since the job was never on the critical path.Same condition as the removed shell branch
The new skip reads exactly the value the old step read, so it narrows nothing:
DIFF_BASEwas empty, withDIFF_BASE: ${{ needs.select-tests.outputs.lane_base }}.lane_baseis${{ steps.base.outputs.ref }}(pr-require-checks.yml:855), and the new row is${{ steps.base.outputs.ref == '' }}, the same output.pr-require-checks.yml:961-965,scripts/resolve-diff-base.sh). Thecheck-testsheader's older wording, "resolves to the whole tree (no usable base, or a manifest change)", predates that rule.scripts/plan-test-lanes.shplans every lane whole for apr-require-checks.ymlchange (whole=1) while the diff base stays set. The old step also returned early there (itsDIFF_BASEwas non-empty), andcheck-pluginsscanned the changed test files.Related
pr-require-checks.ymlchange.🤖 Generated with Claude Code
https://claude.ai/code/session_01KeBVgdbwavfuWZGmKnMhWX