Skip to content

feat(session-flow): drift-check denial kinds and subagent-file gaps in audit-sessions - #10921

Merged
melodic-automation-lanes[bot] merged 1 commit into
mainfrom
work-resolve-issue/issue-10823-40844950764-1
Oct 29, 2026
Merged

melodic-automation-lanes[bot] merged 1 commit into
mainfrom
work-resolve-issue/issue-10823-40844950764-1

Conversation

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor

Closes #10823

Summary

Opened by the work-resolve-issue lane. The run's own summary:

The audit-sessions census now counts each toolDenialKind value as a denial_kind:<value> key, so a denial kind first seen in the newest Claude Code version is reported as a new change by collect.py drift. A new expect-absent canary, invariant:subagent_files_missing, counts sessions that have no subagent file even though an Agent or Task call returned without an error, and it feeds subagents.count. Rejected, denied and failed calls are left out because they write no subagent file.

Not verified

  • The session-flow pytest suite (test_drift.py, test_sweep.py, test_collect.py) was not run directly: this lane runs no interpreter. check-affected passed, but its output does not show whether it ran these suites, so the new and changed tests are left to the PR's CI. The review subagent traced them by
  • tidy was recorded as skipped: passes 1-3 ran (no code edits; one wording edit in the changelog fragment). Pass 4 (/ai-slop:audit fix) ran only as a rubric read and found nothing, because this lane cannot execute the ai-slop detector script, so no detector findings file or fix-pass record was written
  • new-changelog-fragment could not run on the detached HEAD (it names the fragment after the branch), so the fragment was written by hand to match an existing lane fragment. check-changelog-fragments accepts it.
  • The review noted, unverified, that a remote-isolation agent may also leave no local subagent file; a successful remote Agent call would then count toward the invariant. I could not check this against a transcript. Separately, denial_kind keys take part in vanished as effort_value and assistant_err
  • The invariant and denial_kind keys have not been run over a real store, so no current count of either is known; the canaries.json basis says so.
  • The checkout root holds untracked files this run did not create (.bash_profile, .bashrc, .gitconfig, .idea, .mcp.json, .profile, .ripgreprc, .vscode, .zprofile, .zshrc). I left them untouched; they are not part of this change.

Fix

  • plugins/session-flow/skills/audit-sessions/scripts/collect.py: census_keys emits denial_kind:<value> for user records that carry toolDenialKind; new AGENT_TOOLS constant; SessionScan records each Agent or Task call by tool_use id, keeps those whose tool_result is not is_error, and subagent_gap
  • plugins/session-flow/skills/audit-sessions/scripts/census.py: denial_kind added to SECTIONS
  • plugins/session-flow/skills/audit-sessions/reference/canaries.json: new expect-absent canary invariant:subagent_files_missing (record_type assistant, feeds subagents.count) and a dated 2026-10-29 basis note covering it and the denial_kind keys
  • plugins/session-flow/skills/audit-sessions/scripts/tests/test_drift.py: an end-to-end test that collects two versions of transcripts and checks that drift reports denial_kind:fresh as new in 1.0.2; a collector test, run for both Agent and Task, covering a returned call with no file, a call with a fi
  • plugins/session-flow/skills/audit-sessions/scripts/tests/test_sweep.py: subagents.count is no longer expected among the metrics listed unchecked for lack of a canary; tools.denials still is
  • .changes/session-flow/work-resolve-issue-issue-10823-7c2d9a41.md: new minor changelog fragment (tidy also changed one word in it)

Verification

The runner ran each check from the base copy against 92db17d:

  • scripts/check-changelog-fragments.sh --check: passed

The run reports:

  • /home/runner/work/claude-code-plugins/claude-code-plugins/.lane/checks/bin/check-changelog-fragments: all 3 fragments valid (run before tidy, after tidy and after the review fix)
  • /home/runner/work/claude-code-plugins/claude-code-plugins/.lane/checks/bin/check-affected: 12 passed, 6 skipped over 6 changed files (run before and after the review fix)
  • read https: //github.com/melodic-software/claude-code-plugins/issues /6078: the parent issue proposes a denial_kind:<value> census key following effort_value and an expect-absent invariant for sessions with Agent or Task calls and zero subagent files, and reports 16 such sessions in the 2026-1
  • read https: //code.claude.com/docs/en/tools-reference: the page does not say Task was renamed to Agent, so the AGENT_TOOLS comment makes no such claim
  • fresh-context review subagent over the whole diff: no failing-test bug found; it reported one medium finding (Agent calls that are rejected, denied or fail write no subagent file, so normal sessions would trip the invariant). I confirmed and fixed it by counting only calls whose tool_result is not i
  • The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40844950764

Left out: the items under Not verified.

Related

#10823

Lane-Activity: work-resolve-issue/resolve-issue
@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 92db17d906bf1b6978445eb03888230548f9a386: 0 finding(s), 0 as inline comments.

No security issues found in this pull request.

critical-findings: 0

I read all 6 changed files in the full diff and the PR context. The change adds a denial_kind:<value> census key and an invariant:subagent_files_missing canary to the local audit-sessions analytics script. It also adds the matching tests, a canary basis note and a changelog fragment.

  • Trusted paths: none of the changed files match a trusted-path glob. No trust control, token scope, kill switch, merge hold, trusted-actor list, workflow or action is touched.
  • Manifests: no plugin.json or marketplace.json change, so there is no manifest <field> line to report.
  • Instruction surface: nothing is deleted or softened.
  • Data flow: collect.py reads local session transcripts and writes counters to a local census store. The new toolDenialKind value goes through the existing _label helper into a counter key. tool_use_id values are only used as dictionary keys in memory. Nothing is executed, interpolated into a shell or path, sent over the network or posted. The new code adds no secret handling.
  • Embedded instructions: the PR text contains no imperative aimed at the review lane.
  • Spend: this change spends nothing beyond the normal lane run.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 92db17d906bf1b6978445eb03888230548f9a386: 0 finding(s), 0 as inline comments.

No blocking findings. I read the full diff (.claude-lane/pr.diff), the PR context, and the surrounding collect.py and census.py code. I ran no tests, since this lane has no interpreter. Everything the PR says about test results is the author's claim, and I did not verify it.

What I checked

  • denial_kind:<value> keys come out of census_keys through _label, the same path as effort_value and assistant_error. The elif kind == "user" branch is reachable because the earlier branches match other kinds. Adding denial_kind to SECTIONS in census.py matches how the other sections are listed.
  • SessionScan tracks Agent and Task calls by tool_use id. _tool_use records them after its dedup check, and _user drops an id when its tool_result has no is_error. subagent_gap(len(subagents)) runs after all files are scanned, and the count lands in the bucket of the assistant record that made the call. That is the same pattern as usage_regress, and it matches the canary's record_type: assistant.
  • The new test cases cover:
    • an Agent or Task call that returned with no subagent file, which counts;
    • a call with a file, a failed call, and a call with no result, which do not count;
    • the bundled canary degrading subagents.count at 20 of 30 gaps and not at 2 of 30;
    • denial_kind:fresh reported as new in the newest version.
  • The test_sweep.py change drops only subagents.count from the expected-unchecked list. tools.denials stays there, which fits that no canary feeds it.
  • Closing issue feat(session-flow): drift-check denial kinds and subagent-file gaps in audit-sessions #10823, criterion 1 (a new toolDenialKind shows as a "new" change in collect.py drift): met, and test_drift_cli_reports_a_denial_kind_first_seen_in_the_newest_version would fail if it broke.
  • Criterion 2 (an expect-absent invariant for Agent or Task calls with zero subagent files, feeding subagents.count): met, and test_collector_counts_a_returned_agent_call_without_a_subagent_file and test_bundled_subagent_gap_canary_degrades_the_subagent_count cover it.
  • Criterion 3 (a changelog fragment): met by .changes/session-flow/work-resolve-issue-issue-10823-7c2d9a41.md with bump: minor. A new capability under SemVer is a minor bump.
  • No plugin.json or marketplace.json changed, so there are no manifest lines to report.
  • The PR context contains no embedded instructions.

Not blocking, for the author to know

  • The new canary has no measured baseline. canaries.json says outright that neither the subagent_files_missing count nor the 2026-10-03 figure of 16 gap sessions has been checked against a version's min_count. If real sessions normally produce a few gaps, the expect-absent canary could report subagents.count as degraded when nothing is wrong. Run collect.py drift over the reference store once after merge and tune min_count if it fires.
  • The PR notes that an Agent call using remote isolation may leave no local subagent file. Such calls would then count toward the invariant. This is unchecked.

Spend: none that bills. The change only adds local analysis code.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

The unattended merge path armed native auto-merge at head 92db17d906bf1b6978445eb03888230548f9a386.

@melodic-automation-lanes
melodic-automation-lanes Bot added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit d27d8b4 Oct 29, 2026
43 checks passed
@melodic-automation-lanes
melodic-automation-lanes Bot deleted the work-resolve-issue/issue-10823-40844950764-1 branch October 29, 2026 10:31
@github-actions

Copy link
Copy Markdown
Contributor

Post-merge mutation testing

Killed 3, survived 4, timeout 0, no coverage 0, invalid 0, not run 0.
Covered-code score 43%; mutation score 43%.

File Killed Survived Timeout No coverage Invalid Not run
plugins/session-flow/skills/audit-sessions/scripts/census.py 0 1 0 0 0 0
plugins/session-flow/skills/audit-sessions/scripts/collect.py 3 3 0 0 0 0

Surviving mutants

Location Operator Disposition Cause
plugins/session-flow/skills/audit-sessions/scripts/census.py:48 SBR productive input-gap
plugins/session-flow/skills/audit-sessions/scripts/collect.py:270 LCR productive input-gap
plugins/session-flow/skills/audit-sessions/scripts/collect.py:432 LCR productive input-gap
plugins/session-flow/skills/audit-sessions/scripts/collect.py:440 other productive input-gap

The run's summary has the full report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(session-flow): drift-check denial kinds and subagent-file gaps in audit-sessions

0 participants