Skip to content

guardrails: cli-flag-verify PostToolUse hook scans whole file, warns on pre-existing lines outside the edited hunk #543

Closed

Description

@kyle-sexton

Category: behavioral

What happened

During a babysit-prs pass on #529 I edited a single prose line (line 57) in plugins/testing/skills/diagnose/context/loop.md. The cli-flag-verify PostToolUse:Edit hook then flagged dotnet test --project on line 61 — a pre-existing, illustrative code block I did not touch:

cli-flag-verify: 1 unknown flag(s) in ...loop.md
  UNKNOWN_FLAG: dotnet test --project (not found in 'dotnet test --help')

Why it is friction

The hook re-scans the entire file after any edit rather than scoping to the changed hunk. Editing unrelated prose surfaces warnings about flags in lines the change never touched. On docs files that carry deliberately illustrative command snippets, this is recurring noise: every future edit to the file re-raises the same warning, and it competes for attention with warnings that ARE about the current change. (dotnet test --project is also SDK-version-dependent — valid under the .NET 10 MTP-based dotnet test, absent in older VSTest-based dotnet test — so the flag itself is arguable, but that is secondary to the scoping point.)

Suggested direction

Scope cli-flag-verify to the added/changed lines of the edit (the diff hunk) rather than the whole file, so it flags flags the current edit introduced, not pre-existing content. Whole-file scanning still has value for newly-created files; the regression is specifically re-warning on untouched lines during incremental edits.

Context

Filed by an automated PR-babysitting worker (self-observation).

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Jul 19, 2026
  2. kyle-sexton commented on Jul 19, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during triage. · Category: behavioral (bug) · Priority: priority: high (guardrails is pipeline tooling; recurring whole-file noise erodes signal on every incremental edit). Routed status: ready.

    Plan

    cli-flag-verify PostToolUse re-scans the entire file after any Edit, so editing unrelated prose re-surfaces warnings about flags in untouched lines. Diff-scope it.

    Fix: scope the check to the added/changed lines of the edit's hunk — flag only flags the current edit introduced, not pre-existing content. Preserve whole-file scanning for newly-created files (there, every line is "new"); the regression is specifically incremental Edits re-warning on untouched lines.

    Second data point (folded from a near-simultaneous duplicate report)

    Same hook, same root cause, independently hit on a different PR/file: editing toolchain/skills/check/SKILL.md at line 65 fired on dotnet test --project at line 108 (43 lines away, unrelated ecosystem-gotcha prose). Both reports were filed ~3s apart by two babysit workers → the recommend-close-as-duplicate note now on the sibling issue points here.

    Acceptance

    • Editing a line in a file that already contains an unrecognized flag on an untouched line produces NO warning about that untouched line.
    • A flag the edit itself introduces IS still flagged.
    • Newly-created files still get full-body scanning.

    Secondary (separate, do not block the scoping fix)

    Both reports note dotnet test --project may be a false positive in the flag DB — it's valid under .NET 10 MTP-based dotnet test, absent in older VSTest-based. Confirm/whitelist separately; the diff-scoping is the primary fix. Related: #446 (same script, stdin fd0 unbounded read — distinct concern).

    Decision defaulted: diff-scope cli-flag-verify to the edit's changed hunk (keep whole-file scan for new files) — veto before merge

  3. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Jul 19, 2026
  4. kyle-sexton commented on Jul 19, 2026

    @kyle-sexton
    MemberAuthor

    Duplicate of #544 — both filed independently by concurrent babysit workers (#528 and #529) that hit the same cli-flag-verify PostToolUse hook non-diff-scoping bug within seconds of each other, before either's dedupe search could see the other. Leaving both open per babysit's never-close-issues rule; #544 has the fuller reproduction. Flagging the dedupe-race itself as a lane observation, not filing a separate meta-issue for it.

  5. kyle-sexton commented on Jul 20, 2026

    @kyle-sexton
    MemberAuthor

    Claimed by A1 burn lane (main-thread orchestrated), covering the #543/#544 pair: diff-scope the cli-flag-verify PostToolUse check per rule 1 of the just-registered hook-precision convention (PR #761). Repro-first fixture discipline applies. PR to follow; version bump serialized behind in-flight guardrails PRs.

  6. added a commit that references this issue on Jul 21, 2026
    c87a263
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions