You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
performance:snapshot: report and control which code path the measured run took #4437
State left in a plugin data dir by earlier runs (a bench.launched marker) sent every harness sample for the disk-hygiene Stop hook down its Python run path. The hook's common path in real sessions (skip, no interpreter) was never measured by the harness; the result was valid only for the rarer path and nothing in the output said so.
Proposal
Snapshot requires the goal to name the code path(s) under test, and before each arm either resets the state files that select the path or records them, then reports per arm which path ran (for example from a marker, exit code or spawned-process set). An arm whose path does not match the goal is flagged rather than reported as the goal's metric.
Triage: enhancement, verified gap. The goal skill names a measurement boundary and start state (cold / fresh / warm) but nothing names the code path a run must take; the snapshot skill neither resets nor records the state that selects the path, and its report carries no per-arm path. Nothing in the harness-integrity reference covers path-selecting state left by earlier runs. Priority: medium (assessed default; no blocker or deadline). Sibling #4436 touches the same skill but is a separate change (tool version/flags), not a shared decision.
Agent Brief
Type: Feature Summary: Make /performance:goal name the code path under test, and make /performance:snapshot control or record the path-selecting state and report per arm which path ran.
Current behavior:
A goal names metric, targets, floor, boundary and start state. Snapshot captures baseline/post arms with host qualification and interleaving, but a subject whose path is chosen by persisted state (for example a marker file in a plugin data dir left by an earlier run) can run a different path than the goal intends, and the report gives no sign of it. Observed: a stale bench.launched marker sent every sample of a Stop hook down its rare Python path, so the result described the rare path, not the common skip path.
Desired behavior:
The goal records the code path(s) under test, and how to tell which one ran (a marker, exit code, spawned-process set, or other observable).
Before each arm, snapshot either resets the state that selects the path or records it as found; which of the two it did is stated.
The snapshot report states, per arm, the path that ran and the evidence for it.
An arm whose observed path does not match the goal's named path is flagged and not reported as the goal's metric. A path that cannot be observed is reported as unobserved, never assumed.
The harness-integrity checklist gains an item for path-selecting state carried over from earlier runs.
Key interfaces:
/performance:goal output: new required "code path under test" element with its observable.
/performance:snapshot per-arm report: new "path ran" field with evidence, and a mismatch flag.
Harness-integrity reference: a failure-table row and checklist item for stale path-selecting state.
Acceptance criteria:
The goal skill requires naming the code path(s) under test and the observable that identifies each.
The snapshot skill instructs reset-or-record of path-selecting state before each arm and says which was done.
The snapshot report template shows per-arm path and evidence, and a mismatched arm is flagged rather than reported as the metric.
The harness-integrity reference covers stale path-selecting state (table row and checklist item).
Skill bodies follow the repo's skill-body rules (verification records for volatile specifics, ## Next section intact); plugin version and CHANGELOG updated per repo convention.
Reopened. This comes from the 2026-09-29 audit of the unattended Cursor run (report: .work/audit/REPORT.md).
What the audit found.#4820 closed this issue with the goal skill and the harness-integrity reference untouched: 2 acceptance criteria unmet and 1 partial. There is no statement of whether the code path was reset or recorded, and no unobserved outcome for a path that could not be observed. Step 2b also split step 2's closing paragraph. Evidence: .work/audit/units/4437/pr-4820.diff.
What #4820 delivered. A per-arm code-path report in snapshot.
What is still missing on main. The goal skill naming the code path under test, the reset-or-record statement, the unobserved outcome, and the harness-integrity row.
What happens next. Fixed in draft PR #5283: each arm reports a Path: line with reset or recorded state and an unobserved outcome; goal names the code path under test; harness-integrity carries the row; 2b no longer splits step 2. Version and CHANGELOG entries (0.3.0) ship with the PR. The PR is a draft and is not merged.
Activity
kyle-sexton commentedon Sep 27, 2026
This was generated by AI during triage.
Triage: enhancement, verified gap. The goal skill names a measurement boundary and start state (cold / fresh / warm) but nothing names the code path a run must take; the snapshot skill neither resets nor records the state that selects the path, and its report carries no per-arm path. Nothing in the harness-integrity reference covers path-selecting state left by earlier runs. Priority: medium (assessed default; no blocker or deadline). Sibling #4436 touches the same skill but is a separate change (tool version/flags), not a shared decision.
Agent Brief
Type: Feature
Summary: Make
/performance:goalname the code path under test, and make/performance:snapshotcontrol or record the path-selecting state and report per arm which path ran.Current behavior:
A goal names metric, targets, floor, boundary and start state. Snapshot captures baseline/post arms with host qualification and interleaving, but a subject whose path is chosen by persisted state (for example a marker file in a plugin data dir left by an earlier run) can run a different path than the goal intends, and the report gives no sign of it. Observed: a stale
bench.launchedmarker sent every sample of a Stop hook down its rare Python path, so the result described the rare path, not the common skip path.Desired behavior:
Key interfaces:
/performance:goaloutput: new required "code path under test" element with its observable./performance:snapshotper-arm report: new "path ran" field with evidence, and a mismatch flag.Acceptance criteria:
## Nextsection intact); plugin version and CHANGELOG updated per repo convention.Out of scope:
lib/code beyond what reporting the path strictly needs; no new automated path detector.fix(performance): snapshot reports and controls code path per arm (#4437
fix(performance): verify measuring tool version and flags before timi…
fix(performance): require scaling arms for growing-state reads (#4438) (
feat(performance): name the code path in the goal; report path state …
kyle-sexton commentedon Sep 29, 2026
Reopened. This comes from the 2026-09-29 audit of the unattended Cursor run (report: .work/audit/REPORT.md).
What the audit found. #4820 closed this issue with the goal skill and the harness-integrity reference untouched: 2 acceptance criteria unmet and 1 partial. There is no statement of whether the code path was reset or recorded, and no
unobservedoutcome for a path that could not be observed. Step 2b also split step 2's closing paragraph. Evidence:.work/audit/units/4437/pr-4820.diff.What #4820 delivered. A per-arm code-path report in snapshot.
What is still missing on main. The goal skill naming the code path under test, the reset-or-record statement, the
unobservedoutcome, and the harness-integrity row.What happens next. Fixed in draft PR #5283: each arm reports a
Path:line with reset or recorded state and anunobservedoutcome;goalnames the code path under test; harness-integrity carries the row; 2b no longer splits step 2. Version and CHANGELOG entries (0.3.0) ship with the PR. The PR is a draft and is not merged.