Skip to content

fix(ci): stop failing PRs that are only behind main on release notes or a version collision - #7874

Merged
kyle-sexton merged 1 commit into
mainfrom
fix/7871-behind-main-checks
Oct 18, 2026
Merged

kyle-sexton merged 1 commit into
mainfrom
fix/7871-behind-main-checks

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #7871

Summary

A PR that is only behind main should not fail before it reaches the merge queue, which tests every PR on the latest main. This changes two gates:

  • scripts/check-stale-base-overlap.sh warns and exits 0 when every overlapping path is a CHANGELOG.md or a .changes/ fragment. Any other overlapping path still fails. The script runs in the pull_request lane and in the pre-push check-affected run.
  • scripts/check-changelog-parity.sh --check-bump reports VERSION COLLISION and VERSION REGRESSION without failing when CHANGELOG_PARITY_COLLISIONS=advisory is set. Only the pull_request branch of the check-plugins bump step sets it. The merge_group branch does not, so the queue still fails a real collision, and ci-status fails with it (ci-status aggregates check-plugins the same way on both events).

Fix

  • scripts/check-stale-base-overlap.sh: a check after the overlap is computed. When every path is a changelog, it prints a warning that lists the paths and exits 0.
  • scripts/check-changelog-parity.sh: in advisory mode the two monotonicity findings get an ADVISORY prefix and do not count toward the exit code. Every other finding class is unchanged.
  • .github/workflows/pr-require-checks.yml: the pull_request branch of "Verify a version bump updates the plugin's CHANGELOG.md" sets CHANGELOG_PARITY_COLLISIONS=advisory.

Verification

  • scripts/check-stale-base-overlap.test.sh: PASS=10 FAIL=0. The new release-notes-only case fails with the old script (rc=1, STALE BASE). A non-changelog overlap next to changelogs still fails.
  • scripts/check-changelog-parity.test.sh: PASS=108 FAIL=0. The new advisory case fails with the old script (rc=1, VERSION COLLISION). The new merge-queue case (a squash commit on main's 1.1.0, checked against HEAD^1 with no advisory mode) fails as PUBLISHED VERSION REUSE.
  • actionlint and zizmor --offline on the workflow: no findings.
  • scripts/check-affected.sh --tests: the lint checks pass. Two suites fail only because this machine lacks PyYAML and the .github/standards/runner-policy node modules (scripts/pr-bump-plugin-version.test.sh, scripts/run-hygiene-gate.test.sh); CI has both.

Related

  • Gap the brief's scope leaves open: in the last two days, every stale-base failure on a pull_request run that was only behind main overlapped on files other than changelogs. Examples are .github/workflows/pr-unblock.yml, .github/actions/request-lane-token/*, docs/conventions/pr-pipeline.yaml and plugins/guardrails/.claude-plugin/plugin.json. The changelog-only rule does not cover those cases. Covering them means making the CI stale-base step advisory on pull_request, which the merge queue would back up. That gate guards against Stale-base squash merges silently reverted two merged fixes (CI green, issues left closed) #2691, so it is left for a maintainer decision.

🤖 Generated with Claude Code

…or a version collision

check-stale-base-overlap.sh now warns and exits 0 when every overlapping
path is a CHANGELOG.md or a .changes/ fragment; any other overlap still
fails.

check-changelog-parity.sh --check-bump reports VERSION COLLISION and
VERSION REGRESSION without failing when CHANGELOG_PARITY_COLLISIONS is
advisory. The pull_request lane sets it; the merge_group lane does not,
so the queue still fails a real collision and ci-status with it.

Closes #7871

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 18, 2026 01:51
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T01:52:15.449653Z 0a8e4e9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 0a8e4e985480e349d9505445021cda697acdf6df: 0 finding(s), 0 as inline comments.

Security review: PR #7874 at 0a8e4e9

No security findings.

I reviewed all five changed files: pr-require-checks.yml, check-changelog-parity.sh, check-stale-base-overlap.sh, and the two test scripts. The PR relaxes two CI gates on pull_request events:

  • Version collision and regression in check-changelog-parity.sh become advisory when CHANGELOG_PARITY_COLLISIONS=advisory.
  • Stale-base overlap in check-stale-base-overlap.sh warns instead of failing when every overlapping path is a CHANGELOG.md or a .changes/ fragment.

Why neither is a finding:

  • Enforcement is kept. The merge_group branch (pr-require-checks.yml around line 1733) still runs check-changelog-parity.sh --check-bump 'HEAD^1' with no advisory flag. A new test shows a colliding PR still fails there with PUBLISHED VERSION REUSE, so the relaxed pull-request gate is backed by the queue.
  • No new attacker input. The advisory flag is a literal set inline in the workflow step. It is not derived from github.* context, PR text, or any other attacker-controlled value, so no expression injection is added.
  • No new shell injection. The new prefix variable is built from constant strings. The path loop in check-stale-base-overlap.sh uses quoted [[ ... ]] matches and printf '%s', so a hostile filename does not reach a command.
  • No permission or trigger changes. The diff adds no new triggers, permissions: widening or action pins. Running PR-checkout scripts on pull_request was already how this lane worked.

Limits: this is a logic and trust-boundary read of the diff plus a grep of the workflow for the merge-queue path. I did not run the test scripts. The instruction-exception register was not consulted, because the diff deletes no CLAUDE.md, AGENTS.md, rules, skill or hook allowlist content.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 0a8e4e985480e349d9505445021cda697acdf6df: 0 finding(s), 0 as inline comments.

No findings. I read the full diff (5 files) and the surrounding code in scripts/check-stale-base-overlap.sh and scripts/check-changelog-parity.sh. I did not run the scripts or their test suites.

What the PR does

  • scripts/check-changelog-parity.sh: CHANGELOG_PARITY_COLLISIONS=advisory turns VERSION COLLISION and VERSION REGRESSION into prefixed reports that don't increment nonmonotonic. The pull_request lane of .github/workflows/pr-require-checks.yml sets it. The merge-queue branch (--check-bump 'HEAD^1') does not.
  • scripts/check-stale-base-overlap.sh: an overlap made only of CHANGELOG.md files and .changes/ fragments now warns and exits 0. Any other overlapping path still fails.

What I checked

  • Variables: overlap and behind_by are both set before the new block runs.
  • Exit code: nonmonotonic is the only counter the advisory path skips. The final exit decision at check-changelog-parity.sh:920 still counts it everywhere else.
  • Other consumers: nothing else in .github/ or scripts/ greps the "VERSION COLLISION" or "VERSION REGRESSION" strings, so the new prefix breaks no parser.
  • set -e: the [[ ... ]] && prefix=... line is a non-final && list, so it doesn't trip errexit.
  • Merge-queue test: by reading the code, the fixture should still reach PUBLISHED VERSION REUSE. head_version equals fork_version and base_version there, so that check fires before the advisory path can be reached. The test expects exit 1 on that branch.

Minor observation, not a finding
In advisory mode the continue after the collision report skips that plugin's remaining checks for that run (missing release heading, BUMP WITHOUT CHANGE). The PR's comments say the merge queue enforces these. I have no evidence the queue run is stricter than that, so I'm leaving it as a note.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 18, 2026
Merged via the queue into main with commit 1dfa91d Oct 18, 2026
63 checks passed
@kyle-sexton
kyle-sexton deleted the fix/7871-behind-main-checks branch October 18, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(ci): stop failing PRs that are only behind a fast-moving main

1 participant