Skip to content

test(scripts): fail suites that run git while reading the host git config - #17562

Open
kyle-sexton wants to merge 4 commits into
mainfrom
test/17431-fixture-git-global-config
Open

kyle-sexton wants to merge 4 commits into
mainfrom
test/17431-fixture-git-global-config

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #17431

Summary

scripts/check-fixture-git-isolation.sh now has a second rule beside its fixture rule. Every suite that runs git (shell, Python, and JS *.test.{mjs,js,cjs,ts}) must export GIT_CONFIG_GLOBAL and set GIT_CONFIG_NOSYSTEM=1, or GIT_CONFIG_SYSTEM, before its first git command. Otherwise that command reads the host's ~/.gitconfig and system config. The rule applies to fixture-building suites and to every other suite that runs git.

Run against main, the new rule failed 59 suites. This PR fixes all 59, so no allow file is needed.

Fix

  • Ordered verdict. The awk scan now emits per-line events (set, unset, git command, sourced file). Bash replays them, and the gate reports file:line of the first git command that runs without both variables. It catches:
    • a set placed after the first git command (the lane-launcher.test.sh shape);
    • a later unset, including the for v in $(env | sed … GIT_CONFIG_…); do unset loop (the scope-code-files.test.sh shape);
    • an export inside a heredoc body, which earns no credit.
  • Sourced harnesses. A sourced harness counts at its source line, and only when it leaves both variables set. The harness is resolved by path: the suite directory, then its ancestors, then the repository root. If the path does not resolve, the harness counts only when every tracked file with that basename sets both.
  • Python and JS. Module-level code is read in order. A call inside an indented function or test body runs after the module loads, so it is judged against the state at end of file.
  • env= / env: dicts. A git call's env dict is judged on its own:
  • Suites fixed:
    • 7 per-plugin test harnesses now export both, covering 39 suites. The harnesses are architecture, coupling, harness-config (×2), harness-memory, performance and repo-hygiene.
    • 20 more suites now set both themselves. The export keeps any value a runner already set (${GIT_CONFIG_GLOBAL:-/dev/null}). The app-control suites point it at their pinned $HOME.
    • remove-path.test.sh reads ghq.root from a fake HOME, so it now passes GIT_CONFIG_GLOBAL with that HOME.
  • CI. The gate step's changed-file filter and the lint_shell filter group now also match JS test files. Without this, an unisolated .test.mjs would pass its own PR and turn the next shell PR red. scripts/lint-shell-filter.test.sh pins the new match.

Decisions the issue left open:

  • Both variables are required, and GIT_CONFIG_SYSTEM counts as the system half. Basis: scripts/lib/test-git-env.sh:19 exports both. man git-config, ENVIRONMENT, says GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM "Take the configuration from the given files instead from global or system-level configuration".
  • Function bodies are judged against the end-of-file state. Basis: a Python or JS function body runs after module load. Judging by strict line order would flag every helper defined above its export.
  • The opt-out reuses fixture-isolation-scope:, with # or //, rather than a new token. Basis: check-fixture-git-isolation.sh:104-113 already defines that declaration. scripts/lib/test-git-env.sh drops its old unset-only opt-out, which the gate now fails; the gate's failure message names the declaration instead.
  • scripts/fixture-git-global-baseline.txt exists only in the code path. It has the same exact-path and stale-entry semantics as the fixture baseline. No file is shipped, because nothing needs grandfathering.

Verification

  • scripts/check-fixture-git-isolation.test.sh: 80 passed, 0 failed. That count includes 31 new cases for the second rule; the existing passing fixtures now also export both variables.
  • Run against main's checker, the same self-test fails all 16 old-shape cases and passes the 15 fixed-shape cases. This shows the new cases fail on the old shape.
  • scripts/check-fixture-git-isolation.sh --check on this branch: OK (352 isolated, 0 baselined; 389 git-running suites read no host config, 0 baselined). On main the new rule listed 59 violations.
  • The issue's own cases: the new checker, run over the tree before test: isolate fixture git repos from the host's global git config #17416 (fac2a5f^), flags all four suites the issue names. They are lane-launcher.test.sh, scope-code-files.test.sh, test_count_capture_sources.py and test_rank_comment_targets.py.
  • Every touched suite, and every suite that sources a touched harness, was run directly with env -u GIT_CONFIG_GLOBAL -u GIT_CONFIG_NOSYSTEM, so no runner export was present. 112 suites ran:
    • 103 passed on the first run.
    • The three babysit Python suites pass under pytest. The first run had no pytest on its path.
    • That run caught one real regression, in remove-path.test.sh: its git config --global went to /dev/null. It is fixed here and now passes.
    • Five suites fail with the same failure count on the base commit, run the same way, so they are local-host issues rather than this change: permission-state-primary-dir, no-writes, collect-base-activity, graphql-operations (missing graphql module) and block-dangerous-git (strace counts and an shfmt timeout).
  • scripts/lint-shell-filter.test.sh: 18 passed. shellcheck and shfmt report nothing on the checker and its self-test.
  • --check takes 7.1 s here; main's checker took 6.6 s.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd

kyle-sexton and others added 4 commits November 19, 2026 18:37
…nfig

scripts/check-fixture-git-isolation.sh gains a second rule beside the
fixture rule: every suite that runs git (shell, Python, and JS
*.test.{mjs,js,cjs,ts}) must export GIT_CONFIG_GLOBAL and set
GIT_CONFIG_NOSYSTEM=1 (or GIT_CONFIG_SYSTEM) before its first git
command. The verdict is ordered: a set after the first git command, or a
later unset (a loop over GIT_CONFIG_* included), fails and names the
line. A sourced harness counts at its source line, resolved by path.
Python and JS function bodies are judged against the end-of-file state,
and an env= / env: dict that drops both variables fails its call.

The 59 suites on main that failed the new rule are fixed: 7 per-plugin
test harnesses (39 suites) and 20 suites now set both, keeping a value a
runner already exported. The gate step and the lint_shell filter now also start on a JS
test change.

Closes #17431

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
The gate's failure message names the opt-out that now works: a
`fixture-isolation-scope:` declaration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
The classifier grew past the 1000-line size reference once it carried
both rules. The awk program is a separate concern and language from the
bash driver that resolves harnesses and reports, so it moves to
scripts/lib/fixture-git-isolation-scan.awk, run with `awk -f`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 19, 2026 23:53
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T23:55:52.882129Z 50e2b8b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50e2b8b25f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +514 to +515
if (!SH_DATA_ONLY) for (i = 1; i <= n; i++) sh_global_env(segs[i], SH_LN)
sh_global_git(s, SH_LN, SH_DATA_ONLY)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replay shell environment changes in statement order

When multiple statements share a logical line, all their environment changes are emitted before any git-command event. In a synthetic repository, git status; export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 incorrectly passes the gate, while export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1; git status; unset GIT_CONFIG_GLOBAL incorrectly fails. This both misses host-config reads and rejects isolated commands. Emit environment and git events together for each statement instead of processing all environment changes first.

Useful? React with 👍 / 👎.

Comment on lines +201 to +204
if (s ~ ("environ[ \t]*\\[[ \t]*" Q "GIT_CONFIG_GLOBAL" Q "[ \t]*\\][ \t]*=[^=]") ||
s ~ ("(environ[ \t]*\\.[ \t]*setdefault|setenv|putenv)[ \t]*\\([ \t]*" Q "GIT_CONFIG_GLOBAL" Q) ||
((s ~ /environ[ \t]*\.[ \t]*update[ \t]*\(/ || s ~ /patch[ \t]*\.[ \t]*dict[ \t]*\([ \t]*(os[ \t]*\.[ \t]*)?environ/) && index(s, "GIT_CONFIG_GLOBAL"))) g_ev("S", ln, "G")
if (s ~ ("environ[ \t]*\\[[ \t]*" Q "GIT_CONFIG_NOSYSTEM" Q "[ \t]*\\][ \t]*=[ \t]*" Q "?(1|true|yes)") ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude unexecuted Python function assignments from module state

Assignments inside Python function bodies currently emit unconditional process-environment events, so merely defining a helper can make an unisolated module-level git call pass. I reproduced this with an uncalled def unused() that assigns both config variables, followed by a module-level subprocess.run(["git", "status"]): the gate reports success even though neither assignment executes. Track function-body mutations separately from module-level state so only executed initialization can credit the module-level call.

Useful? React with 👍 / 👎.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 50e2b8b25f47c8237ebcb5e2e8b7fa54325b20e7: read 49 of 49 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #17562 (head 50e2b8b)

No security issues found.

What the change does. It adds a second rule to the test-hygiene gate scripts/check-fixture-git-isolation.sh: any suite that runs git must export GIT_CONFIG_GLOBAL and set GIT_CONFIG_NOSYSTEM=1 (or GIT_CONFIG_SYSTEM) before its first git command. The awk scanner moves into scripts/lib/fixture-git-isolation-scan.awk and gains event output that bash replays in order. About 30 suites and harnesses now export both variables, and the CI filters now also match JS test files.

What I checked

  • Trusted path .github/workflows/pr-require-checks.yml. I read the whole change:

    • L1112–L1131 add scripts/fixture-git-global-baseline.txt and **/*.test.{mjs,js,cjs,ts} to the lint_shell changed-file filter.
    • L2150–L2157 add the same patterns, the new awk file and the new baseline to the gate step's git diff --name-only pathspec.

    Both only add paths that start the existing check. Nothing changes in permissions:, triggers, if: conditions, token requests or ci-status hold logic. $DIFF_BASE is still read from the environment, not interpolated into the run: script. More pull requests now run the gate, and no gate is narrowed.

  • Trust controls. No change to check-trusted-trigger, request-lane-token, check-kill-switch, resolve-config, the merge-hold scripts or babysit risk/verify logic. The three changed babysit files are tests (tests/test_babysit_*.py, tests/test_merge_holds_sync.py). Each only adds module-level os.environ.setdefault("GIT_CONFIG_GLOBAL", os.devnull) and GIT_CONFIG_NOSYSTEM=1. The merge-related JS tests (merge-job.test.mjs, upstream-apply-changelog.test.mjs) get the same environment change and no assertion changes.

  • Gate scripts. check-fixture-git-isolation.sh and the new awk file only read tracked file contents and print tab-separated records. The unquoted ${GEV[...]} word-splitting and the g_resolve path normalization run on repository content inside an advisory lint. They only look up IS_TRACKED and never execute or source the resolved path, so nothing there can be injected. The checker's own git ls-files calls are unchanged in kind.

  • Instruction-surface lens. scripts/lib/test-git-env.sh drops a three-line comment about an unset-based opt-out for reading the host git config. That is test-harness documentation, not a guardrail; the fixture-isolation-scope: declaration replaces it. docs/contracts/instruction-exception-register/README.md was not consulted: the change removes no standing instruction from CLAUDE.md, AGENTS.md, rules, skills or hooks.

  • Direction of the change. Tests that now point GIT_CONFIG_GLOBAL at /dev/null, or at a fake HOME under mktemp -d, read less of the host's config (signing, hooks, URL rewrites), not more.

  • Other files. The 15 .changes/*.md fragments are bump: none notes. No plugin.json or marketplace.json changed, so there are no manifest lines.

  • PR context. The description holds no embedded instructions to the lane.

Spend note: JS-test-only pull requests now also run the lint-shell job, which uses a little more standard Actions time on the existing runner. This is not a hold.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 2 inline finding(s) on 50e2b8b25f47c8237ebcb5e2e8b7fa54325b20e7; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment on lines +116 to +117
if (r ~ /environ|process[ \t]*\.[ \t]*env/)
return (r ~ /GIT_/ ? "fresh" : "inherit")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A fresh env dict that reads one value from the process environment counts as inherit, so it passes. env_rhs returns inherit for any expression that mentions environ or process.env but not GIT_. The shapes meant to inherit (**os.environ, ...process.env, os.environ.copy(), dict(os.environ, …)) are already caught by the branch above. So this branch only changes the verdict for a literal that reads a value, and such a literal is a new, smaller environment.

Here is a JS input that passes the gate, while git reads the host's real ~/.gitconfig and /etc/gitconfig:

process.env.GIT_CONFIG_GLOBAL ??= "/dev/null";
process.env.GIT_CONFIG_NOSYSTEM = "1";
test("x", () => {
  execFileSync("git", ["status"], { env: { PATH: process.env.PATH, HOME: process.env.HOME } });
});

The Python version passes too: subprocess.run(["git", "status"], env={"PATH": os.environ["PATH"], "HOME": os.environ["HOME"]}). Both go call_env → env_rhs → inherit → an F event → checked against the end-of-file state 3 → pass.

This misses the issue's criterion "fails a suite that runs git with neither … (including env= dicts)" and the PR's "a dict built fresh … without both variables fails". The self-tests miss it because g-fresh.test.mjs names GIT_CONFIG_NOSYSTEM, so its dict contains GIT_.

Fix: remove this branch and let a {…} / dict(…) literal with no spread fall through to fresh. A non-literal expression that mentions the environment can stay inherit. Then add a self-test case for { PATH: process.env.PATH } and {"PATH": os.environ["PATH"]} with want=fail.

claude-lane: claude-review

Comment on lines +319 to +323
g_eval "$f" "$3"
((G_STATE == 3)) || {
GHARNESS["$key"]=0
break
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A sourced harness gets credit even when it runs git before its own export. g_harness checks only G_STATE == 3, the harness's end state, and ignores the G_BAD that g_eval just computed for it. Harness files (test-helpers.sh, lib/*.sh) are never scanned as suites, because suites at :358 is *.test.sh / test_*.py / *_test.py. So nothing ever checks their own ordering.

Here is an input that passes the gate. A harness lib/h.sh contains:

REPO_TOP="$(git rev-parse --show-toplevel)"
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1

A suite then runs . "$(dirname "$0")/h.sh" followed by git. The harness's git rev-parse reads the host config, which is the lane-launcher.test.sh shape the issue names, moved into a harness. The suite still gets state=3 at its source line, and no violation is reported. A harness B event (a git call handed a dict that drops both) is ignored the same way.

Fix: require [[ -z "$G_BAD" ]] beside G_STATE == 3 before granting credit. Alternatively, report the harness's G_BAD line against the suite that sources it. Then add a self-test case with this harness shape and want=fail.

claude-lane: claude-review

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 50e2b8b25f47c8237ebcb5e2e8b7fa54325b20e7: read 49 of 49 changed file(s); 2 finding(s), 2 as inline comments.

Code review: PR #17562, a second rule for check-fixture-git-isolation

The PR adds an ordered, per-line global/system-config rule to scripts/check-fixture-git-isolation.sh. The awk scan moves to scripts/lib/fixture-git-isolation-scan.awk and emits events that bash replays to name the first git command that runs unisolated. The PR also fixes 59 suites, 39 of them through 7 harnesses, and widens the CI filters so JS test files trigger the gate. The fixes are mechanical and consistent: every touched suite or harness exports both variables before its first git call. The one deliberate override is the app-control suites' pinned $HOME/.gitconfig. remove-path.test.sh correctly passes GIT_CONFIG_GLOBAL with its fake HOME. The refactor leaves the first rule's logic as it was.

There are two blocking findings, both in the checker. Each one lets a suite pass even though its git commands read host config.

  1. A fresh env dict that reads PATH from the process environment counts as "inherit". See the inline comment on scripts/lib/fixture-git-isolation-scan.awk:116-117. { PATH: process.env.PATH, HOME: process.env.HOME } and env={"PATH": os.environ["PATH"]} both pass, but they drop both variables. This misses part of the issue's first Done-when criterion: "The checker fails a suite that runs git with neither GIT_CONFIG_GLOBAL set nor GIT_CONFIG_NOSYSTEM=1, in shell, JS and Python (including env= dicts)." It also contradicts the PR's own claim that "a dict built fresh … without both variables fails". The self-tests miss it: the only fresh-dict case (g-fresh.test.mjs) names GIT_CONFIG_NOSYSTEM, so it contains GIT_.
  2. A sourced harness gets credit even when it runs git before its own export. See the inline comment on scripts/check-fixture-git-isolation.sh:319-323. Harnesses are never scanned as suites (:358), so the lane-launcher.test.sh shape the issue names goes unflagged once it moves into a harness.

Closing issue #17431, the other criteria:

  • Late set or later unset: met. Tests: g-late, g-unset, g-loop, test_g_late.py, test_g_pop.py, g-delete.test.mjs.
  • Self-tests in check-fixture-git-isolation.test.sh: present. The PR author says the old checker fails the new old-shape cases; I did not run that.
  • The first criterion: the shell, Python and JS cases are covered, except for the env-dict gap in finding 1.

The PR's verification numbers (80 passed, 389 suites, 112 direct runs) come from the author. This lane has no shell, so I ran nothing.

Loosened-check verdict: fail (default; the prompt has no loosened_check_verdict: line); patterns: 0 (the prompt has no loosened_check_patterns: line). The prompt also has no loosened_check_decision_record: line, so the citation requirement was not checked. No change in the diff loosens a check. The gate gets stricter, the CI filters get wider, and the existing self-test fixtures only gain exports their expected outcomes now require.

Spend (not a hold): adding **/*.test.{mjs,js,cjs,ts} to the lint_shell filter group starts the lint-shell job on PRs that only touch JS tests. That uses standard Actions minutes.

The PR description and the closing issue contain no embedded directives aimed at the lane.

No verifier subagent ran: I traced both findings by hand through the awk scan and g_eval / g_harness.
Verifier dropped 0 of 2 candidates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(scripts): make check-fixture-git-isolation catch suites that read global git config

1 participant