Repository navigation
test(scripts): fail suites that run git while reading the host git config - #17562
kyle-sexton wants to merge 4 commits into
Conversation
…nfig
scripts/check-fixture-git-isolation.sh gains a second rule beside the
fixture rule: every suite that runs git (shell, Python, and JS
*.test.{mjs,js,cjs,ts}) must export GIT_CONFIG_GLOBAL and set
GIT_CONFIG_NOSYSTEM=1 (or GIT_CONFIG_SYSTEM) before its first git
command. The verdict is ordered: a set after the first git command, or a
later unset (a loop over GIT_CONFIG_* included), fails and names the
line. A sourced harness counts at its source line, resolved by path.
Python and JS function bodies are judged against the end-of-file state,
and an env= / env: dict that drops both variables fails its call.
The 59 suites on main that failed the new rule are fixed: 7 per-plugin
test harnesses (39 suites) and 20 suites now set both, keeping a value a
runner already exported. The gate step and the lint_shell filter now also start on a JS
test change.
Closes #17431
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
The gate's failure message names the opt-out that now works: a `fixture-isolation-scope:` declaration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
The classifier grew past the 1000-line size reference once it carried both rules. The awk program is a separate concern and language from the bash driver that resolves harnesses and reports, so it moves to scripts/lib/fixture-git-isolation-scan.awk, run with `awk -f`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 50e2b8b25f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!SH_DATA_ONLY) for (i = 1; i <= n; i++) sh_global_env(segs[i], SH_LN) | ||
| sh_global_git(s, SH_LN, SH_DATA_ONLY) |
There was a problem hiding this comment.
Replay shell environment changes in statement order
When multiple statements share a logical line, all their environment changes are emitted before any git-command event. In a synthetic repository, git status; export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 incorrectly passes the gate, while export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1; git status; unset GIT_CONFIG_GLOBAL incorrectly fails. This both misses host-config reads and rejects isolated commands. Emit environment and git events together for each statement instead of processing all environment changes first.
Useful? React with 👍 / 👎.
| if (s ~ ("environ[ \t]*\\[[ \t]*" Q "GIT_CONFIG_GLOBAL" Q "[ \t]*\\][ \t]*=[^=]") || | ||
| s ~ ("(environ[ \t]*\\.[ \t]*setdefault|setenv|putenv)[ \t]*\\([ \t]*" Q "GIT_CONFIG_GLOBAL" Q) || | ||
| ((s ~ /environ[ \t]*\.[ \t]*update[ \t]*\(/ || s ~ /patch[ \t]*\.[ \t]*dict[ \t]*\([ \t]*(os[ \t]*\.[ \t]*)?environ/) && index(s, "GIT_CONFIG_GLOBAL"))) g_ev("S", ln, "G") | ||
| if (s ~ ("environ[ \t]*\\[[ \t]*" Q "GIT_CONFIG_NOSYSTEM" Q "[ \t]*\\][ \t]*=[ \t]*" Q "?(1|true|yes)") || |
There was a problem hiding this comment.
Exclude unexecuted Python function assignments from module state
Assignments inside Python function bodies currently emit unconditional process-environment events, so merely defining a helper can make an unisolated module-level git call pass. I reproduced this with an uncalled def unused() that assigns both config variables, followed by a module-level subprocess.run(["git", "status"]): the gate reports success even though neither assignment executes. Track function-body mutations separately from module-level state so only executed initialization can credit the module-level call.
Useful? React with 👍 / 👎.
|
claude-security-review reviewed Security review: PR #17562 (head 50e2b8b)No security issues found. What the change does. It adds a second rule to the test-hygiene gate What I checked
Spend note: JS-test-only pull requests now also run the lint-shell job, which uses a little more standard Actions time on the existing runner. This is not a hold. critical-findings: 0 |
| if (r ~ /environ|process[ \t]*\.[ \t]*env/) | ||
| return (r ~ /GIT_/ ? "fresh" : "inherit") |
There was a problem hiding this comment.
A fresh env dict that reads one value from the process environment counts as inherit, so it passes. env_rhs returns inherit for any expression that mentions environ or process.env but not GIT_. The shapes meant to inherit (**os.environ, ...process.env, os.environ.copy(), dict(os.environ, …)) are already caught by the branch above. So this branch only changes the verdict for a literal that reads a value, and such a literal is a new, smaller environment.
Here is a JS input that passes the gate, while git reads the host's real ~/.gitconfig and /etc/gitconfig:
process.env.GIT_CONFIG_GLOBAL ??= "/dev/null";
process.env.GIT_CONFIG_NOSYSTEM = "1";
test("x", () => {
execFileSync("git", ["status"], { env: { PATH: process.env.PATH, HOME: process.env.HOME } });
});The Python version passes too: subprocess.run(["git", "status"], env={"PATH": os.environ["PATH"], "HOME": os.environ["HOME"]}). Both go call_env → env_rhs → inherit → an F event → checked against the end-of-file state 3 → pass.
This misses the issue's criterion "fails a suite that runs git with neither … (including env= dicts)" and the PR's "a dict built fresh … without both variables fails". The self-tests miss it because g-fresh.test.mjs names GIT_CONFIG_NOSYSTEM, so its dict contains GIT_.
Fix: remove this branch and let a {…} / dict(…) literal with no spread fall through to fresh. A non-literal expression that mentions the environment can stay inherit. Then add a self-test case for { PATH: process.env.PATH } and {"PATH": os.environ["PATH"]} with want=fail.
claude-lane: claude-review
| g_eval "$f" "$3" | ||
| ((G_STATE == 3)) || { | ||
| GHARNESS["$key"]=0 | ||
| break | ||
| } |
There was a problem hiding this comment.
A sourced harness gets credit even when it runs git before its own export. g_harness checks only G_STATE == 3, the harness's end state, and ignores the G_BAD that g_eval just computed for it. Harness files (test-helpers.sh, lib/*.sh) are never scanned as suites, because suites at :358 is *.test.sh / test_*.py / *_test.py. So nothing ever checks their own ordering.
Here is an input that passes the gate. A harness lib/h.sh contains:
REPO_TOP="$(git rev-parse --show-toplevel)"
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1A suite then runs . "$(dirname "$0")/h.sh" followed by git. The harness's git rev-parse reads the host config, which is the lane-launcher.test.sh shape the issue names, moved into a harness. The suite still gets state=3 at its source line, and no violation is reported. A harness B event (a git call handed a dict that drops both) is ignored the same way.
Fix: require [[ -z "$G_BAD" ]] beside G_STATE == 3 before granting credit. Alternatively, report the harness's G_BAD line against the suite that sources it. Then add a self-test case with this harness shape and want=fail.
claude-lane: claude-review
|
claude-review reviewed Code review: PR #17562, a second rule for
|
Closes #17431
Summary
scripts/check-fixture-git-isolation.shnow has a second rule beside its fixture rule. Every suite that runs git (shell, Python, and JS*.test.{mjs,js,cjs,ts}) must exportGIT_CONFIG_GLOBALand setGIT_CONFIG_NOSYSTEM=1, orGIT_CONFIG_SYSTEM, before its first git command. Otherwise that command reads the host's~/.gitconfigand system config. The rule applies to fixture-building suites and to every other suite that runs git.Run against main, the new rule failed 59 suites. This PR fixes all 59, so no allow file is needed.
Fix
file:lineof the first git command that runs without both variables. It catches:lane-launcher.test.shshape);unset, including thefor v in $(env | sed … GIT_CONFIG_…); do unsetloop (thescope-code-files.test.shshape);sourceline, and only when it leaves both variables set. The harness is resolved by path: the suite directory, then its ancestors, then the repository root. If the path does not resolve, the harness counts only when every tracked file with that basename sets both.env=/env:dicts. A git call's env dict is judged on its own:GIT_*, without both variables fails. This is thetest_count_capture_sources.pyshape from before test: isolate fixture git repos from the host's global git config #17416.${GIT_CONFIG_GLOBAL:-/dev/null}). The app-control suites point it at their pinned$HOME.remove-path.test.shreadsghq.rootfrom a fake HOME, so it now passesGIT_CONFIG_GLOBALwith that HOME.lint_shellfilter group now also match JS test files. Without this, an unisolated.test.mjswould pass its own PR and turn the next shell PR red.scripts/lint-shell-filter.test.shpins the new match.Decisions the issue left open:
GIT_CONFIG_SYSTEMcounts as the system half. Basis:scripts/lib/test-git-env.sh:19exports both.man git-config, ENVIRONMENT, saysGIT_CONFIG_GLOBALandGIT_CONFIG_SYSTEM"Take the configuration from the given files instead from global or system-level configuration".fixture-isolation-scope:, with#or//, rather than a new token. Basis:check-fixture-git-isolation.sh:104-113already defines that declaration.scripts/lib/test-git-env.shdrops its old unset-only opt-out, which the gate now fails; the gate's failure message names the declaration instead.scripts/fixture-git-global-baseline.txtexists only in the code path. It has the same exact-path and stale-entry semantics as the fixture baseline. No file is shipped, because nothing needs grandfathering.Verification
scripts/check-fixture-git-isolation.test.sh: 80 passed, 0 failed. That count includes 31 new cases for the second rule; the existing passing fixtures now also export both variables.scripts/check-fixture-git-isolation.sh --checkon this branch:OK (352 isolated, 0 baselined; 389 git-running suites read no host config, 0 baselined). On main the new rule listed 59 violations.fac2a5f^), flags all four suites the issue names. They arelane-launcher.test.sh,scope-code-files.test.sh,test_count_capture_sources.pyandtest_rank_comment_targets.py.env -u GIT_CONFIG_GLOBAL -u GIT_CONFIG_NOSYSTEM, so no runner export was present. 112 suites ran:remove-path.test.sh: itsgit config --globalwent to/dev/null. It is fixed here and now passes.permission-state-primary-dir,no-writes,collect-base-activity,graphql-operations(missinggraphqlmodule) andblock-dangerous-git(strace counts and an shfmt timeout).scripts/lint-shell-filter.test.sh: 18 passed. shellcheck and shfmt report nothing on the checker and its self-test.--checktakes 7.1 s here; main's checker took 6.6 s.Related
🤖 Generated with Claude Code
https://claude.ai/code/session_01KeBVgaQE4QB75SPD3ky1jd