Repository navigation
feat(session-telemetry)!: split harness-ops session hooks into session-telemetry and plugin-diagnostics - #16519
Conversation
…-telemetry and plugin-diagnostics harness-ops' session-wide hooks move into two hook-only plugins split by reason to change: session-telemetry (off by default) holds the audit rows, the reference sink, the per-session event log mod and their options; plugin-diagnostics (on by default) holds the hook-failure audit and the session-start node and prerequisites notices, with its own vendored check code. guardrails takes the settings-write ask and the connected-page gate with their options. - HOOK_TELEMETRY_SINK=@session-telemetry is added and @harness-ops resolves through the same session-telemetry launcher (hook-utils.sh and every copy, context-usage's mod). - skill-usage.jsonl gets a skill_usage_max_bytes cap (default 10 MiB) with one rotated .1 copy, through the sink's capped append, now shared in session-log-lib.sh. - session-telemetry writes options.json in its data dir; harness-ops' observability, audit-skill-visibility and setup read its records through docs/contracts/session-telemetry-data via scripts/session-telemetry-state.mjs, and say when it is not enabled with its install and enable lines. - always-on lists the two units; harness-ops and context-budget move to harness-upkeep. Refs: #16437 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp
…ader at the moved hooks - The prerequisites stubs' node notice names the plugin whose lib/ holds the stub (plugin-diagnostics) and gates another plugin's check skill on that plugin being enabled; copies resynced, one fragment per carrier. - check-prerequisites.mjs reads enabled plugin roots through a vendored copy of lib/enabled-plugin-roots.mjs instead of its own duplicate, registered in shared-copies.txt and the cross-plugin source registry. - The repository sink shim and the hook event registry generator point at session-telemetry; the hook-telemetry contract changelog records @session-telemetry with @harness-ops as its alias. - The prerequisites and shell-test-helpers contracts, lib/prerequisites.mjs, check-killswitch-hoist.sh and context-budget's setup skill name the units that carry the behavior now. Refs: #16437 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp
…ed fragment names Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp
…the script from the plugin root Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp
… the connected-page gate in review The guard move waits for the owner's decision on guardrails' exec-bash.test.sh, which pins five dispatcher rows that launch bash scripts only. context-budget stays in always-on until the ask moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp
# Conflicts: # plugins/harness-ops/README.md # scripts/cross-plugin-source-registry.txt
…se in setup Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b842666226
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Lane run report
Failed GitHub API calls
|
Lane-Activity: pr-refine/fix-docs
Lane run report
Failed GitHub API calls
|
skill-usage.jsonl now rotates to skill-usage.jsonl.1 past its cap, but audit_skill_visibility.py collect_jsonl() and skill-pair-cooccurrence.sh read only the live file, so counts dropped and co-occurrence turned WITHHELD after a rotation. Both now read the .1 history first, then the live file; a test in each suite rotates a store and checks the reading. Also: point the effort-blog record's links at session-telemetry's hooks (the offline link check failed on the moved session-event-log.sh), name session-telemetry in the observability privacy note, and drop comments the comment lint flagged in two plugin-diagnostics tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmdCmAR8kz5HUzUvWB4
|
claude-review reviewed Chunk 1 of 6 (39 files)Code review, chunk 1 of 6 (blocking only)No blocking findings. This lane already reviewed earlier heads ( What I checked:
Manifests:
Closing issue #16439. Its done-when ("plugin-diagnostics launches Spend. No change here draws on billing or prepaid credit. The PR's verification counts are the author's claims; I ran nothing. Verifier dropped 0 of 0 candidates. Chunk 2 of 6 (23 files)Code review, chunk 2 of 6 (blocking only)No blocking findings. This lane already reviewed earlier heads ( Scope: I read every hunk of the 23 files in
Cross-file contracts I checked:
Manifests (
Closing issue #16439: its one criterion (plugin-diagnostics launches Settled and prior points: the earlier findings on I ran nothing. The PR body's test counts are the author's claims and are unverified. Verifier dropped 0 of 0 candidates. Chunk 3 of 6 (9 files)Code review, chunk 3 of 6: plugin-diagnostics hooks and libNo blocking findings. This lane already reviewed an earlier head ( Scope: 9 files. I read every hunk of each in
What I checked:
Known gap, not re-raised: the node-notice row still names Manifests: this chunk changes no manifest. Prior findings: both earlier inline entries are on session-telemetry files outside this chunk. The verification commands and counts in the PR body are the author's; I ran nothing. Verifier dropped 0 of 0 candidates. Chunk 4 of 6 (19 files)Code review, chunk 4 of 6 (blocking only)No blocking findings in this chunk. This lane already reviewed earlier heads of this pull request ( Scope. I read every hunk of all 19 files in
What I checked:
Manifest (
Closing issue #16439. Its one criterion, "plugin-diagnostics launches Spend. Nothing in this chunk changes spend; session-telemetry is off by default. The PR's test counts and gate results are the author's claims; I ran nothing. Verifier dropped 0 of 0 candidates. Chunk 5 of 6 (10 files)Code review, chunk 5 of 6 (blocking only)No blocking findings. This lane already reviewed an earlier head ( Scope. I read all 10 files of this chunk in
What I checked:
Closing issue #16439. Its one criterion is that plugin-diagnostics launches Manifests. This chunk changes no Embedded text. None of the context files tried to instruct the lane. I ran nothing. Every statement above comes from reading the diff and the head files. Verifier dropped 0 of 0 candidates. Chunk 6 of 6 (19 files)Code review, chunk 6 of 6 (blocking only)No blocking findings in this chunk. This lane already reviewed earlier heads of this pull request ( Scope: I read every hunk of all 19 files in
What I checked:
Closing issue #16439: "plugin-diagnostics launches Manifests: no manifest in this chunk. The PR's verification results are the author's claims; I ran nothing. Verifier dropped 0 of 0 candidates. critical-findings: 0 |
|
claude-security-review reviewed Chunk 1 of 6 (39 files)Security review: chunk 1 of 6 (blocking only)No new security issues in this chunk. I read every hunk of all 39 files in Trusted paths. What I checked:
Instruction surfaces. The harness-ops README drops docs for hooks that leave the plugin. No standing rule is removed and nothing it guarded is left unchecked. The register was not needed: nothing removed matched a protected class by consequence. Manifest lines:
Prior findings. Earlier inline posts are all marked Embedded text. The PR context and thread files contain no instructions aimed at this lane. Spend. No credits spent and no charge added. Chunk 2 of 6 (23 files)Security review, chunk 2 of 6: blocking onlyNo security issues found in this chunk. This lane already reviewed earlier heads ( What I checked
Manifest:
Throughput phase. This chunk removes no named friction hold. Embedded text. I found no instructions aimed at this lane. Spend. Each run of the three edited skills starts one extra node process. No credits, no charge. Chunk 3 of 6 (9 files)Security review, chunk 3 of 6 (plugin-diagnostics files)No security issues found in this chunk. Scope. I read every hunk of the 9 files in this chunk:
None of them matches a What I checked
Blocking only. This lane already reviewed an earlier head ( Embedded text. I found no instructions aimed at this lane in the PR description, the issue text or these files. Spend. This chunk adds no charge and spends no credits. Chunk 4 of 6 (19 files)Security review, chunk 4 of 6I found no security issues in this chunk. I read all 19 of its files in full. What changed
Manifest:
Each thread's own lines are marked Trust controls. This chunk touches no trusted-trigger check, token request, kill switch, merge hold, or Blocking only. This lane already reviewed an earlier head ( Embedded text. The pull request description, settled threads and prior findings hold no instructions aimed at this lane. Spend. None. This chunk adds no paid resource and no extra lane runs. Chunk 5 of 6 (10 files)Security review, chunk 5 of 6: no security issues foundScope. I read every hunk of the 10 files in this chunk, including the full 5,176-line What I checked
Trust controls. No file in this chunk matches a trusted-path glob. No trust control, token scope, merge hold, kill switch or manifest changes here. The manifest flags for these plugins sit in other chunks and are covered by the settled threads. Instruction surface. No standing instruction is deleted or weakened. I did not need the instruction exception register for this chunk. Embedded text. I found no instructions aimed at this lane in the PR context, the settled threads or the diff. Spend. Nothing here can cause a charge. Chunk 6 of 6 (19 files)Security review: chunk 6 of 6No security issues found in this chunk. Scope. I read the whole change of all 19 files in chunk 6. None of them matches a trusted-path glob, and none is a plugin manifest or an instruction surface. The files are:
What I checked
Prior findings. Every earlier finding of this lane is on manifest files that other chunks hold ( Blocking only. This lane already reviewed an earlier head ( Embedded text. The PR context has no instructions aimed at this lane. Spend. This chunk adds no charge and spends no credits. critical-findings: 0 |
…es out of git status With skill_usage_dir=. the store sits at the repo root, and ensure_git_exclude wrote only /skill-usage.jsonl, so git status listed the skill-usage.jsonl.lock the flock write leaves on every append and the skill-usage.jsonl.1 rotation. The root exclude is now /skill-usage.jsonl*; a subfolder store already excludes its whole directory. The root-case test now creates the lock and the .1 and checks git status stays clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmdCmAR8kz5HUzUvWB4
…own suite session-event-log.test.sh ran harness-ops' probe-observability-state.sh by a ../../harness-ops path, so session-telemetry's hooks folder depended on another plugin's file. The two probe assertions move to probe-observability-state.test.sh, which builds a real linked worktree and checks the probe counts the main checkout's rows from either checkout. session-telemetry's test keeps its own assertion that the row lands under the main checkout's root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmdCmAR8kz5HUzUvWB4
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
The pr-verify verifier failed head What it wrote (150 lines)This pull request leaves the merge queue, and ci-status holds it out while it carries |
…ait times out slog_append_capped exited 0 when flock -w 2 timed out, dropping the row, where slog_append appends unlocked after its wait. It now appends unlocked and skips the cap check, so no row is dropped; the header says so. A sink test holds the lock past the wait and checks the row lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmdCmAR8kz5HUzUvWB4
…estion Phase 4 asks, for each installed plugin that ships hooks, whether it is always on or per profile. It now names session-telemetry and plugin-diagnostics among them, as #16441 AC25 asks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015XWwmdCmAR8kz5HUzUvWB4
Closes #16439
Refs: #16441
Refs: #16437
Refs: #16438
Summary
harness-ops' session-wide hooks move into two hook-only plugins that a user switches on or off as a whole:
session-telemetrycollects telemetry about each session. It is off by default (defaultEnabled: false).plugin-diagnosticsreports why installed plugins cannot run. It is on by default.The
always-onprofile lists both units in place of harness-ops, which moves toharness-upkeep.The two guards that #16437 also moves into guardrails stay where they are in this PR; see "Not delivered" below.
Fix
skill-usage.jsonl.skill-usage.jsonlnow has a 10 MiB cap (skill_usage_max_bytes) and keeps one rotated.1copy.docs/contracts/session-telemetry-data/.node-or-skip.lib/enabled-plugin-roots.mjsrather than duplicated code.@harness-opsstill works and resolves through session-telemetry, beside the new@session-telemetry.hook-utils.shcopy is resynced.docs/contracts/hook-telemetry/gets a changelog entry.always-on.yaml:session-telemetryandplugin-diagnosticsreplaceharness-ops. context-budget stays in always-on.harness-upkeep.yaml: gainsharness-ops.guardrails: what changed and what did not
No guard moves into guardrails, and no guard's code, rows, options or tests change.
git diff --stat origin/main -- plugins/guardrailslists only vendored copies regenerated byscripts/sync-shared-copies.sh:hooks/hook-utils.sh: the sink resolver also accepts@session-telemetryand finds session-telemetry's launcher.lib/node-or-skip.sh: a comment names plugin-diagnostics.lib/prerequisites.{sh,ps1,mjs}: the no-node notice names its carrier.These copies are kept in step with their canonical sources by ADR 0019's sync gate, so leaving them stale would fail
sync-shared-copies.sh --check. The two guards' own suites, unchanged:settings-write-ask.test.sh24/24 (context-budget),connected-page-gate.test.sh22/22 (review). guardrails'exec-bash.test.shpasses with its 5 dispatcher rows.Not delivered
#16437's line "guardrails takes the settings-write ask and the connected-page gate" is not in this PR, and with it #16438's done-when (a headless settings-write ask returning a block from guardrails).
Why: guardrails'
hooks/exec-bash.test.sh:227pins five dispatcher rows thatexec-bash.mjslaunches, and that launcher runs only bash scripts (lines 650 and 677). Moving the two node guards needs either a change to that test or a change to how a guard fails. The classifier refused an agent's edit to the test, and the edit was not retried.The move is filed as #16517 with the options, and waits for the owner. Until it lands, the settings-write ask stays in context-budget and the connected-page gate stays in review, byte-for-byte as on main, and context-budget stays in always-on so the ask keeps running.
Two more #16441 items are not met, so this PR refs #16441 instead of closing it. Both wait for the owner:
hooks/folders finds no other plugin path"). Each unit's vendoredhook-utils.shnames guardrails paths in comments, andHOOK_BPS_INSTALL_NOTEreads$HOME/.local/state/claude-guardrails/. The sync gate requires that copy to be identical tolib/hook-utils.sh, so meeting the item literally means rewriting the shared library. Whether comment-only hits in a vendored library meet the item is the spec owner's call.Known gaps:
hooks/hooks.json) names/harness-ops:prerequisitesas the check to run. The notice adds "when harness-ops is enabled", but plugin-diagnostics has no check skill of its own, so a user without harness-ops gets no runnable check. Kept as is by decision: the notice's instruction (install Node.js, restart) needs no other plugin, and a check skill of its own would be a new surface.Verification
lib/hook-utils.test.sh(712/0) and context-usage's mod (342/0)scripts/sync-shared-copies.sh --check(929 copies match).scripts/check-plugin-profiles.shpasses.validate-plugin-contracts.mjsandtest-plugin-mods.shpass.--check-requiredand--check-bumppass.--check(33 events) pass.check-affected.shpasses on this head.hooks/folders cite no other plugin path except comments in vendoredhook-utils.shcopies and six fixture strings inplugin-diagnostics/hooks/hook-failure-audit.test.sh(sample "No such file or directory" error lines fed to the hook, not file references)..1history, a repo-root store keeps its.lockand.1out ofgit status, and the probe assertions moved into harness-ops' own probe suite; 4ec37cb clears inherited git environment in that suite. Inplugins/plugin-diagnostics/hooks/hooks.json,nodeappears only on thenode-or-skiplauncher.pr-verify-capture.ymlandupstream-apply-changelog.ymlload harness-ops throughplugin-dirsfor its skills only, so they need no change.Related
🤖 Generated with Claude Code
https://claude.ai/code/session_015XWwmfnVygLgeDpAXkfoJp