Skip to content

context-budget 0.6.10 — post-use audit: additivity claim, discarded caveats, hook scope, and a measured deny counterexample #3356

Closed
3 of 3 issues completed

Description

@kyle-sexton

Plugin: context-budget@melodic-software v0.6.10 · Measured against: Claude Code 2.1.241,
C:\Users\KyleSexton\.local\bin\claude.exe · Mode: sdk (exact), headless · Platform:
Windows 11, cwd $HOME (nonrepo state key) · Agent SDK 0.3.241

Audited after a real end-to-end run: /context-budget:setup check + apply, then
/context-budget:audit --full-sweep (27 tools + additivity), then an applied lever with a
before/after ledger row. Three components in scope: the audit skill, the setup skill, and the
settings-write-ask PreToolUse hook.

Severities are the auditors' own, uncalibrated (no human calibration was applied).


HIGH

H1 — --verify-additivity cannot detect non-additivity on a full sweep

combinedSaved for the observed run was 24,197 — exactly 10854 + 13343, the entire attributed
payload. systemBucketSaving (measure.mjs:610-614) sums both bucket deltas, and a combined deny
drives both to zero, so on any full sweep the check degenerates to sumOfParts === baselineTotal.
It reports additive: false whether or not anything is wrong.

Reproduced: denying only the 11 prefix savers leaves 13 live tools including ToolSearch (worth
+15,824 prefix when denied) and the SDK still omits the bucket, which the engine records as a
measured 0.

Fix: add a saturation guard — additive: null with a reason when either bucket is a synthesized
zero, plus an epsilon. Report additivity per bucket; the data is already on every perTool row.

H2 — SKILL.md's "deltas add" is false as written

SKILL.md:18 claims A/B differencing "is compositional (deltas add), so a basket of trims can be
priced from its members", unconditionally. Non-additivity is real, established from three clean
non-saturated readings: baseline 10,557; denying the 10 prefix savers other than PowerShell →
2,399 (−8,158, shortfall exactly zero); PowerShell solo → −3,355. PowerShell stays live in the
10-deny run, so its marginal contribution is bounded by the 2,399 remaining while its solo value is
3,355 — at least 956 tokens double-counted, with no synthesized zero anywhere in that argument.

Deferred-side additivity, by contrast, is exact: 12 deferred savers summed to 13,343 (the bucket, to
the token), and a 3-tool basket predicted 4,608 and measured 4,608.

The skill-frontmatter-subtraction hypothesis is dead: compareSnapshots requires both sigMatch
and skillTokensMatch (measure.mjs:551-558, 588), so a constant subtraction cancels in a delta;
independently confirmed 1:1 — a 297-token skill moved Skills +297 and System tools −297.

Fix: qualify the claim per bucket in both homes — SKILL.md:18 and
levers.json → deny-bare-tool.categoryBasis (SSOT duplication). Ship with H1 or it is cosmetic.

H3 — measure.mjs:723 discards every deny run's caveats

attribute returns caveats: baseline.caveats, so caveats raised by any deny run — including the
synthesized-zero disclosure promised at engine.md:76-79 — never reach the record. This is what
made H1 shippable
: the instrument's own disclosure channel is severed.

Fix: merge deny-run caveats into the attribution record. Smallest diff, highest value in this
report — do this one first regardless of the rest.

H4 — measured counterexample to deny-bare-tool.mechanism (possibly UPSTREAM)

With a bare-name permissions.deny entry for EndConversation in force, the tool remained in
the interactive session's deferred registry, ToolSearch select:EndConversation returned its full
schema, and the call succeeded (the session ended). Verified verbatim against the permissions page
(WebFetch + curl, 498,122 bytes, 2026-08-24): "Bare-name deny rules like Bash remove the tool
from Claude's context before this evaluation begins
"
, and the options table's "The Bash tool
definition is removed from the request. Claude does not see the tool and cannot attempt it."
No
carve-out for any tool appears there.

The same mechanism demonstrably works on this machine for other deferred tools: 27 → 24 live,
deferred bucket −4,608, exact. So this is tool-specific, not general breakage.

The row's recheckTrigger is structurally blind to it. It reads "a measured bare-name deny stops
moving the System tools buckets", but EndConversation never enters either bucket headless — the
trigger could never fire for this case. That is a defect in the trigger's design, not a missing row.

⚠️ Likely upstream. The plugin cites the docs accurately; the docs and the binary disagree.
Please do not "fix" a correct citation. Consider escalating to Anthropic.

Fix: add EndConversation to interactive-only-tools.json; record the deny-no-op as a measured
counterexample; widen deny-bare-tool.recheckTrigger to cover tools appearing in neither attributed
bucket.

H5 — apply's scope guidance is refuted on both branches

setup's apply says to "pass the scope claude plugin list reports for this plugin … or the write
lands at a scope that does not load." Both branches fail: project-scope pluginConfigs is ignored
outright per docs, or the write goes to user settings regardless and the -s/cwd ceremony is
inert. It also omits that -s governs enabledPlugins placement, so following it can install the
plugin at a new scope and create a stray <cwd>/.claude/settings.json.

Observed trigger: with cwd == $HOME, claude plugin list reported this plugin twice (user
and project) because project scope resolves to <cwd>/.claude/settings.json — byte-identical to
the user path. One file, two rows. The auditing session initially acted on that and gave the operator
wrong advice.

Fix: replace with doc-grounded facts — default user; -s governs enablement placement;
plugin list can show one file under two labels. Resolve the disjunction before finalizing wording.

H6 — check step 1 cannot detect the failure its own rationale describes

Step 1 argues the hook registers in exec form ("command": "node"), so a bare node must resolve
in the hook's environment, and that an unresolvable node means the checkpoint silently never
fires while the option still reads true. But the prescribed probe is command -v node, which under
fnm returns a per-call ephemeral multishell path (AppData\Local\fnm_multishells\<pid>_<ts>\node
— three probes in one session returned PIDs 64452, 64196, 33540). That is neither the hook's
environment nor the persisted PATH. The check PASSes and stops.

The Windows caveat that would catch this ("confirm the hook's environment resolves the same node")
is filed under the Missing node bullet — reachable only on FAIL. The failure it describes is a
PASS-path failure.

Fix: on Windows, resolve against persisted Machine+User PATH, report in-process resolution
separately, flag PID-bearing shim paths as ephemeral, FAIL when only an ephemeral hit exists.
Ambitious version: a check --deep that fires the PreToolUse path against a throwaway
settings-shaped path and confirms the ask actually arrives.


MEDIUM

M1 — hook matcher cannot see shell writes, but the claim says "any settings-surface write"

Matcher is Write|Edit|MultiEdit|NotebookEdit. A PowerShell Set-Content or a Bash heredoc to
~/.claude/settings.json bypasses the checkpoint entirely, so SKILL.md:224's "any settings-surface
write" and the header's "the ordinary auto-mode path cannot rewrite settings silently" are both
overclaimed. The bypass enumeration names two exotic escapes (PermissionRequest, disableAllHooks)
and omits the reachable one.

Found independently by two auditors, in separate packets. Corroborated in-session: an inline
node -e write to a settings-composing file was stopped by the auto-mode classifier, while the
Edit path — the one the matcher covers — went through.

Fix: extend the matcher to Bash|PowerShell, or keep it and reword the claim. Do not ship
the current claim with the current matcher. (Extending has real false-positive cost; honest scoping
is likely the better fix.)

M2 — the hook does not cover render-into-settings sources (chezmoi class)

The regex (settings-write-ask.mjs:45-46) matches only .claude/settings.json,
.claude/settings.local.json, and managed-settings.json. Four Edit calls landed on
~/.local/share/chezmoi/.chezmoidata/claude.json — which provably composes the deployed
~/.claude/settings.json (chezmoi source-path → dot_claude/modify_settings.json, which consumes
it) — with no ask.

Reranked up on evidence: .claude is a protected directory and .mcp.json/.claude.json are
protected files, so on most paths this hook matches it largely duplicates harness protection. The
in-cwd non-protected file (this class) is the one write path with no harness review at all.
managed-settings.json is on neither protected list and is the hook's only genuinely non-redundant
coverage.

Fix: at minimum document the scope boundary. A general solution is hard (arbitrary templating),
but the class is real and recognizable.

M3 — disable-artifact is offerable but structurally unmeasurable

The row is removes-weight / recommendable-on-fit, yet its own measurement field prescribes
attribute --tools Artifact, and Artifact is in interactive-only-tools.json — never a headless
candidate. Both halves of the route are dead inside the skill's declared scope. Applied live here,
the Skills delta was 0 because the artifact-* skills never enter a headless listing.

knownUncovered protects tools from exactly this misreading; there is no lever-side
equivalent
. report.md's "Zeros are findings" then requires publishing the zero — so the contract
actively causes a working lever to be reported as inert.

Fix: add a lever-side unmeasurable-in-this-session-kind marker plus a report.md rule
rendering such levers in their own group rather than as zeros.

M4 — knownUncoveredRecord attributes operator-caused absence to product structure

It computes listing.tools − candidates (measure.mjs:634-641), so AskUserQuestion,
EnterPlanMode, and ExitPlanMode — absent because the operator bare-name-denied them — are
reported under "interactive-only — structurally unreachable."

Fix: subtract known deny entries before attributing absence to interactive-only status.

M5 — works-but-saves-nothing-here asserts a consumer-specific fact statically

skillOverrides / disable-bundled-skills are stored with that category, conditional on cap
saturation. Measured here the cap was slack (13/13 included, 0 collapsed, 2,111 tokens against a
~30,000 budget at skillListingBudgetFraction: 0.03), so the condition resolves the opposite way and
the levers would genuinely remove weight. Separately the saturation signal those rows name is absent
from the record and unit-mismatched (tokens vs characters).

Fix: split into stored mechanism + runtime-resolved saving; note the token-vs-character unit
change wherever the listing budget appears.

M6 — meta.purpose claims completeness that verify-catalogue cannot check

The catalogue claims to hold "every known operator-controllable switch", but verify-catalogue is
one-directional: it confirms stored tokens exist in the binary and cannot see existing-but-unstored
switches. Binary scan of 2.1.241 found CLAUDE_CODE_DISABLE_CRON and CLAUDE_CODE_ENABLE_DESIGN_SYNC
with no rows. (RemoteTrigger / PushNotification genuinely have no native switch — same scan — so
deny is correct for those.)

Fix: add --find-unstored to verify-catalogue; ENV_TOKEN_RE is already the right regex.

M7 — SDK data is fetched and thrown away

The SDK returns per-skill tokens (summing exactly to the Skills bucket) and slashCommands;
measure.mjs:396 discards both. Fix: keep skillFrontmatter[].tokens + pluginName; record
slashCommands in the snapshot.

M8 — a headless artifact is attributed to bypassPermissions

SKILL.md stamps the hook's block behavior as measured at v2.1.232 under bypassPermissions. The
measurement is not wrong; the attribution is imprecise. Re-verified this audit at 2.1.241 across
three headless probes — default, auto, and bypassPermissions — all fired and blocked, no file
created. Fix: re-stamp to 2.1.241 and scope the claim to what was actually exercised.

M9 — custom-output-style detection has no built-in-style branch

Detection reads "keep-coding-instructions in the style's frontmatter", but a built-in style has no
frontmatter file. Confirmed live: outputStyle: "Concise" set, ~/.claude/output-styles/ absent.

M10 — apply prints a write-causing command without the never-run marker

The network-only npm install carries an explicit "operator's to run" marker; the
claude plugin install … --config command — which writes pluginConfigs — does not. An agent
executing apply can therefore cause exactly the write the skill's own contract forbids.


LOW

  • L1 — nonrepo cwd additionally drops project settings, .mcp.json, and CLAUDE.md discovery;
    nothing states the reported total is a floor. State it in report.md §1.
  • L2 — --full-sweep (argument-hint) vs --tools from-baseline (engine flag); disclosed at
    SKILL.md:101, cosmetic.
  • L3 — the "uninstall drops the entire stored pluginConfigs entry" claim carries no version
    stamp while the --config claim two sentences earlier does — inconsistent evidence discipline
    inside one paragraph.
  • L4 — step 4's template renders "Report the effective true": the sentence has no noun.
    One-word fix, not a malformed template.
  • L5 — the printed npm remediation is POSIX shell handed to a Windows operator to run themselves.
  • L6 — the skill names an SSOT then restates all three prerequisites inline anyway (drift
    surface).

OPEN QUESTION — not a finding, and no remediation depends on it

During this session, four settings-composing Edit calls and one probe write produced no ask,
in an interactive auto-mode session. In fresh headless sessions the hook fires and blocks in all
three permission modes tested. Two hypotheses remain, and nothing runnable from here separates them:

  • H-1a — post-/reload-plugins --force session state. That reload reported 1 hook reloaded and
    1 error during load (unattributed; /plugin is operator-only). Of the two enabled plugins only
    context-budget ships a hook, so 1 hook can only be this one. The anomalous counter is
    0 skills against 4.
  • H-1b — an interactive dispatch gap. hooks.md L1732 documents that a hook's "ask" forces a
    prompt in auto mode, and records a prior bug of this exact shape (pre-2.1.211, Bash only, not
    Write). If this reproduces, it is an upstream Claude Code defect, not a plugin one.

An auto-mode-only gap (H-1c) was tested and disconfirmed: headless auto mode fires normally.

Closing test: a fresh interactive session in auto mode writing to any
managed-settings.json outside .claude/. Manual mode no longer discriminates.


What worked (recorded so it is not regressed)

The engine refused a number for Skill and published the subtraction-artifact reason instead;
preserved CronDelete/CronList zeros rather than dropping them; caveats[] honestly empty;
verify-catalogue returned absent: [] across all 19 rows; the dual-ledger prefixDelta /
deferredDelta split held throughout and is what made the ToolSearch finding legible (deny it and
13,343 tokens migrate into the window, +15,824 prefix). setup apply was genuinely idempotent
and wrote nothing. The disableWorkflows lever was correctly detected as already applied. The ledger
round-trip predicted 4,608 and measured 4,608, exact.


Evidence packets (sealed, with per-file digests) at
~/.claude/plugins/data/plugin-quality-melodic-software/evidence/6cd8f3c9-.../<target>/20260824T203113Z/
— audit-notes.md per target, plus evidence*.md and contract.md.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Aug 24, 2026
  2. claude commented on Sep 5, 2026

    @claude
    Contributor

    Generated by Claude Code

  3. claude commented on Sep 5, 2026

    @claude
    Contributor

    This was generated by AI during triage.

    Triage: partly aged out → re-verify, then decompose

    Lane: autonomous triage, instance vm-triage-202609050304. Claimed via the tracker seam.

    Read this before picking up any finding here

    The audit was run against context-budget v0.6.10. Current main ships v0.6.22. At least one HIGH finding has been fixed in the interim and at least one is still live. The severity table cannot be trusted as a work list without per-finding re-verification. This triage re-checked the HIGH tier only.

    HIGH-tier re-verification against current main

    H1 — --verify-additivity cannot detect non-additivity on a full sweep. FIXED (substantially).
    The saturation guard H1 asked for now exists. systemBucketSaving() (measure.mjs:608-612) detects vanished buckets and returns saved: null rather than a synthesized zero, with the source comment stating the contract explicitly: "callers publish that as incomparable with vanished naming the buckets, never coerce it to 0." The --verify-additivity block (:680-703) consumes it — it computes comparable = cmp.comparability.systemToolsComparable && !vanished.length and attaches vanishedReason(vanished) to reasons.

    Two deltas from what H1 asked for, both minor and both worth a small child rather than reopening the finding:

    • additive is still a boolean (comparable && combinedSaved === sumOfParts) rather than null when incomparable. The information is not lost — it rides in comparable: false plus reasons — but a caller reading only additive still sees false where it should see "unmeasured".
    • Per-bucket additivity reporting was not implemented. H1's second half stands: the data is already on every perTool row.

    H2 — SKILL.md's "deltas add" is false as written. STILL LIVE.
    plugins/context-budget/skills/audit/SKILL.md still reads, unconditionally: "which is compositional (deltas add), so a basket of trims can be priced from its members." No per-bucket qualification was added. The finding's own evidence is strong and does not depend on a synthesized zero — three non-saturated readings put at least 956 tokens of double-counting on the prefix side, while the deferred side was exact to the token (12 savers summing to 13,343; a 3-tool basket predicting and measuring 4,608). H2 also correctly records that the skill-frontmatter-subtraction hypothesis is dead, confirmed 1:1.

    H3 — measure.mjs:723 discards every deny run's caveats. LIKELY FIXED; line reference has drifted.
    The cited line is now unrelated code. Caveats are carried through at measure.mjs:721 (caveats: baseline.caveats) and are appended in several places (:388, :505, :866-868). Whether the deny run's own caveats specifically survive was not established here and is the one thing a child must confirm before assuming this is closed.

    H4, H5, H6 — NOT re-verified by this triage. H4 is flagged in the body as possibly UPSTREAM, which changes its routing entirely.

    Why this is not briefed as one item

    Six HIGH + ten MEDIUM + a LOW tier + an open question is not one contract. The findings span an engine correctness bug, two SSOT documentation defects, hook-matcher scope, several catalogue-metadata claims, and an upstream-attribution question. Beyond size, the aging problem makes a single brief actively harmful: it would hand a cold agent a work list of which some entries are already done.

    Correct next action: /work-items:decompose, with a mandatory re-verification step per finding.

    Decomposition plan

    Step 0, blocking, for the decompose run. Before creating any child, re-verify each finding against v0.6.22 and drop the ones already fixed. H1 and H3 are the known cases; there may be more in the MEDIUM tier, which this triage did not check. Record each verdict in-thread so the next reader inherits it.

    Then create children by cluster, not one per finding — several of these share one contract:

    # Cluster Findings Work class Exit
    1 The additivity claim, both homes H2, plus H1's unimplemented per-bucket half and the additive-should-be-null nuance work-class: scoped delegable
    2 Catalogue claims that assert consumer-specific or unmeasurable facts statically M3, M4, M5, M6 work-class: scoped decision-defaulted
    3 Hook matcher scope M1, M2 — the matcher cannot see shell writes or render-into-settings (chezmoi-class) sources, while the claim says "any settings-surface write" work-class: scoped human-gated — narrowing a claim vs widening a hook is a threat-model call, and widening an always-on hook is governed by .claude/rules/hook-budget.md
    4 apply / check guidance refuted by measurement H5, H6 work-class: scoped delegable
    5 Upstream counterexample H4 work-class: read-only first human-gated — if it really is upstream, this becomes an upstream report, not a repo change
    6 Remaining MEDIUM/LOW M7, M8, M9, M10 and the LOW tier work-class: mechanical likely delegable

    Cluster 1 is the one to land first. H2 is a false claim in a shipped instruction surface, it is duplicated across two homes (SKILL.md:18 and levers.json → deny-bare-tool.categoryBasis), and the correct wording is already established by measurement: prefix-side deltas do not add; deferred-side deltas do. Both homes must change together — this repo treats SSOT duplication as a defect in its own right.

    Do not create a child for the OPEN QUESTION section. The body marks it as not a finding with no remediation depending on it. Leave it in the container.

    Classification of this container

    • Raw marker cleared. priority: medium applied. The severities are the auditors' own and explicitly uncalibrated (no human calibration was applied), and the highest-severity finding verified still-live (H2) is a documentation correctness defect rather than a runtime fault. Recorded per convention: reporter tier HIGH, uncalibrated.
    • work-class: read-only — this item is an audit report; its remaining work is re-verification plus decomposition.
    • Not agent-ready. Handing this to an autonomous lane as filed would produce work on already-fixed findings.

    Lease released on completion of this triage.


    Generated by Claude Code

  4. removed their assignment
    on Sep 5, 2026
  5. kyle-sexton commented on Sep 6, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during triage.

    Decomposition executed, honoring the plan's blocking step 0

    The plan posted here on 2026-09-05 was re-verified and executed. Instance vm-triage-202609050304.

    Re-verification (2026-09-06) against the current version

    The plugin has advanced again since the plan was written. Re-checked directly:

    • The additivity claim is still live, at both homes, and the plan's diagnosis was exact. The skill body still asserts unconditionally that the measure is compositional and deltas add. The lever catalogue's entry for the same lever independently asserts that deltas are additive and a basket prices from its members. Neither carries any qualification. Two surfaces, one false fact.
    • The saturation guard is present and correct, with the source comment stating the contract the finding asked for and the verification path consuming it. That half is genuinely fixed.
    • The per-bucket half and the boolean-verdict nuance are still unimplemented, as the plan predicted.
    • The hook-matcher scope findings are still live. The checkpoint registers against file-editing tools only while the skill body claims coverage of any settings-surface write. Confirmed by reading both.

    Children created

    Child Cluster Exit Class
    #3863 The additivity claim in both homes, plus the per-bucket and boolean remnants delegable scoped
    #3864 Hook-matcher scope versus the coverage claim human-gated scoped
    #3865 Step 0: re-verify the audit's unchecked findings delegable read-only

    Why only three, and why one of them is the re-verification itself

    The plan opens with a step marked blocking: re-verify every finding against the current version and drop the ones already fixed, before creating any child. That instruction is the most valuable thing in the plan, because this audit has now demonstrably decayed — of the findings checked so far, one is fixed, one is partly fixed, and one is live.

    Creating children for the remaining clusters would have violated that step, and would have produced exactly the harm it exists to prevent: work items describing defects that may no longer exist, handed to agents who will find something to change regardless. Re-verifying all sixteen remaining findings properly was more than this pass could do with the care the parent's own evidence deserves.

    So the remaining clusters were not sliced. Instead #3865 carries step 0 as a scoped, read-only, delegable work item, with instructions to record a verdict per finding and to verify by behavior rather than by line reference — the audit's line references have already drifted once, and one cited line now points at unrelated code. A second decompose pass on this container turns the survivors into work.

    The two clusters that were sliced are the ones this pass re-verified first-hand, so they rest on observation rather than on the audit's word.

    Note on #3863

    It is the item to land first, as the plan says. Both homes must change in one PR: this repository treats a fact duplicated across two surfaces as a defect in itself, so correcting one and leaving the other is the same defect with a new instance. The corrected wording must state which side composes and which does not, rather than softening into vagueness; "may not always add" would be true and useless.

    Not sliced, deliberately

    The open-question section, per the plan. It is marked as not a finding, with no remediation depending on it, and stays in the container.

    This container stays open and was not modified beyond its labels.

    Triage claim released by vm-triage-202609050304.


    Generated by Claude Code

  6. kyle-sexton commented on Sep 27, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during work-loop.

    Step 0 re-verification (#3865): verdicts for every finding not already re-checked

    Checked against: context-budget 0.6.31 at main e509cb7b2. Binary scans used Claude Code 2.1.283 (~/.local/share/claude/versions/2.1.283). The pass was read-only and changed no files.
    Method: each finding was checked against its named behavior or concept in the current source. The audit's line references were not used to locate anything. The line references below are current as of e509cb7b2.
    Out of scope, already tracked elsewhere: H1 and H2 (#3863), M1 and M2 (#3864), and the OPEN QUESTION section.

    18 findings verified. 15 still reproduce (H4 only on the plugin side), and 3 are fixed.

    Finding Verdict Observed evidence (current file:line)
    H3: deny-run caveats discarded Reproduces, and is worse than the audit framed it attribute publishes caveats: baseline.caveats (skills/audit/scripts/measure.mjs:789). The per-tool deny runs (:729-731) and the combined run (:750) never have their .caveats read. In sdk mode sdkSnapshot zero-fills an omitted attributed bucket and pushes a caveat naming the synthesized zero (:385-393). That caveat is therefore the only disclosure: because the bucket is filled with 0, the null/vanished path in systemBucketSaving (:619-628) never fires in sdk mode. The record breaks the promise in reference/engine.md:94-97 that "a caveats[] entry names every synthesized zero".
    H4: bare-name deny of EndConversation is a no-op Split verdict. Plugin side: reproduces. Upstream side: cannot determine Plugin side: EndConversation appears nowhere in the plugin (grep), including reference/interactive-only-tools.json:4-10. deny-bare-tool.recheckTrigger is unchanged (reference/levers.json:48) and can still only fire on movement in the two System tools buckets. Upstream side: the docs still say a bare-name deny rule removes the tool from Claude's context (code.claude.com/docs/en/permissions.md line 99, fetched this pass), and the 2.1.283 binary still carries EndConversation. Whether the deny is still a no-op was not tested: it needs a fresh interactive session with permissions.deny: ["EndConversation"] followed by ToolSearch select:EndConversation. Routing: if that reproduces, the docs/binary disagreement goes to an upstream report to Anthropic, not to a change here. The recheck-trigger blind spot and the missing interactive-only listing entry route here regardless of the upstream result.
    H5: scope guidance refuted on both branches Reproduces (moved) apply was removed in 0.6.16. The same guidance now sits in check's remediation block: "-s defaults to user; pass the scope claude plugin list reports … or the write lands at a scope that does not load" (skills/setup/SKILL.md:89-92). It still does not say that -s governs enabledPlugins placement, or that plugin list can show one file under two labels. The source of truth is shared: the setup text restates caveat 2 of docs/conventions/plugin-reconfiguration/README.md:41-43, which every setup skill prints. The fix belongs in the convention, not only in this plugin.
    H6: check step 1 cannot detect its own failure Reproduces Step 1 still probes command -v node (skills/setup/SKILL.md:33) and still argues the exec-form, hook-environment risk (:37-42). The Windows caveat "confirm the hook's environment resolves the same node" still appears only under the FAIL-path "Missing node" bullet (:70-72). Nothing handles persisted PATH or PID-bearing ephemeral shim paths.
    M3: disable-artifact offerable but unmeasurable Reproduces The row is still removes-weight / recommendable-on-fit, and its measurement still prescribes attribute --tools Artifact (reference/levers.json:104). Artifact is still interactive-only (reference/interactive-only-tools.json:5). No lever-side unmeasurable marker exists (grep: no "unmeasurable" in the catalogue or report.md). "Zeros are findings" is unchanged (reference/report.md:59-60).
    M4: operator-denied tools reported as interactive-only Reproduces knownUncoveredRecord returns listing.tools − candidates with the fixed reason "interactive-only — structurally unreachable" (measure.mjs:701-708). The candidates are [...tools, ...baseline.tools] (:785-788), and no deny entries are subtracted. AskUserQuestion, EnterPlanMode and ExitPlanMode are still on the list (interactive-only-tools.json:6,8,9).
    M5: works-but-saves-nothing-here asserted statically Reproduces, with the instruction side mitigated skill-overrides and disable-bundled-skills still store that category (levers.json:190, :213). The instruction side already routes around it: audit SKILL.md:129-132 says "resolve conditions by measurement" and report.md:42-43 requires "the condition's measured resolution". Still missing: the data-model split into a stored mechanism plus a runtime-resolved saving; a collapsed-row count in the snapshot (only totalSkills/includedSkills/tokens are recorded, measure.mjs:417-423); and any note on the token-versus-character unit (grep: none).
    M6: completeness claim that verify-catalogue cannot check Reproduces meta.purpose still reads "every known operator-controllable switch" (levers.json:4). runVerifyCatalogue still iterates only stored rows (measure.mjs:886-955), and there is no --find-unstored. On 2.1.283, CLAUDE_CODE_DISABLE_CRON and CLAUDE_CODE_ENABLE_DESIGN_SYNC both occur in the binary and neither has a row.
    M7: SDK data fetched and thrown away Reproduces skillFrontmatter.map((s) => ({ name: s.name, source: s.source })) still drops tokens and pluginName (measure.mjs:395-396). slashCommands is never recorded (grep: no occurrence).
    M8: headless artifact attributed to bypassPermissions Fixed (trivial remnant) The audit SKILL.md:235-246 now scopes the claim to "a headless run that skips permissions". It is re-verified 2026-09-06 against 2.1.263 and three cited pages. Remnant: the hook header comment still carries the old "v2.1.232 … under bypassPermissions" stamp (hooks/settings-write-ask.mjs:8-12). This is a comment, not an instruction surface.
    M9: custom-output-style has no built-in-style branch Reproduces Detection still reads "keep-coding-instructions in the style's frontmatter" (levers.json:395) and has no branch for a built-in style that has no file.
    M10: write-causing command printed without the never-run marker Fixed apply was removed in 0.6.16. The claude plugin install … --config line now sits under "Remediation guidance (printed by check; the operator applies it)" (skills/setup/SKILL.md:63), and the section states "This skill never writes user settings or pluginConfigs" (:92-93). The skill has no action that executes it.
    L1: nonrepo total is a floor, and nothing says so Reproduces report.md:18-19 stamps "the working directory measured from" but has no floor statement. lib/state-key.sh:177 still resolves the nonrepo rung.
    L2: --full-sweep hint versus the from-baseline flag Reproduces (cosmetic, disclosed) The argument hint says [--full-sweep] (skills/audit/SKILL.md:3), and the body maps it to --tools from-baseline (:107). The engine has no --full-sweep flag (measure.mjs:154).
    L3: uninstall claim unstamped next to a stamped --config claim Reproduces (relocated) The setup skill now delegates both claims to the reconfiguration convention (skills/setup/SKILL.md:81-93). That convention's verified-version record stamps only the --config still-writes claim, at 2.1.240 (docs/conventions/plugin-reconfiguration/README.md:30-36). Its uninstall-drops-pluginConfigs caveat has no stamp (:40-42). The same inconsistency now lives in the shared source of truth.
    L4: "Report the effective true" has no noun Reproduces skills/setup/SKILL.md:56 still reads "Report the effective ${user_config.settings_write_ask_enabled}", which still renders to "Report the effective true".
    L5: POSIX npm remediation handed to a Windows operator Reproduces Both copies are still POSIX-only mkdir -p … && npm install --prefix …: skills/setup/SKILL.md:77-78 and skills/audit/SKILL.md:64-65.
    L6: SSOT named, then prerequisites restated inline Reproduces skills/setup/SKILL.md:24-26 names the audit's Prerequisites and the measure.mjs header as the source of truth, then restates node, the CLI and the SDK in :33-55. That is a third copy after skills/audit/SKILL.md:56-66.

    Which clusters still have live findings

    Cluster numbers follow the 2026-09-05 decomposition plan.

    • Cluster 2 (catalogue claims): all four live. M3, M4, M5 and M6. For M5, only the data-model half remains.
    • Cluster 4 (apply/check guidance): both live. H6 is plugin-local. H5's wording is shared with docs/conventions/plugin-reconfiguration/README.md, so its slice has to touch the convention, and through it every setup skill, not just context-budget.
    • Cluster 5 (H4): live on the plugin side. The listing entry and widening recheckTrigger can be sliced here now. The upstream half stays human-gated and read-only until someone runs the interactive test. If it reproduces, it becomes an upstream report, not a repo change.
    • Cluster 6 (remaining medium and low): M7, M9, L1, L2, L4, L5 and L6 are live. L3 is live but belongs with the convention change in cluster 4. Drop M8 and M10. M8's hook-comment remnant is a one-line comment fix and does not need its own finding.
    • H3 is in none of the plan's six clusters. It is live and, from the evidence above, the most consequential engine defect remaining. Give it its own slice (scoped, delegable). Do not let the second decompose pass drop it.
    • Cluster 1 (context-budget: correct the "deltas add" claim in both homes and finish the per-bucket additivity reporting #3863) and cluster 3 (context-budget: the settings-write checkpoint claims to cover any settings write but only sees file-edit tools #3864) were not re-checked in this pass.

    This container stays open for the second decompose pass. No children were created and nothing was fixed.

  7. 3 remaining items

  8. added
    spec-containerSpec container: sub-issues are its work items (/work-items:decompose and /work-items:ship).
    and removed on Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    spec-containerSpec container: sub-issues are its work items (/work-items:decompose and /work-items:ship).

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions