Skip to content

feat(harness-ops): route hook telemetry to the reference sink via @harness-ops - #14279

Merged
kyle-sexton merged 3 commits into
mainfrom
feat/14227-harness-ops-sink-launcher
Nov 9, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
feat/14227-harness-ops-sink-launcher

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #14227

Summary

A repository can now send its hook telemetry to harness-ops' reference sink without copying the sink into the repository and without a per-machine path. It sets "HOOK_TELEMETRY_SINK": "@harness-ops" in its settings.json. A copy was needed before because settings env values are copied as written, with no shell expansion (env-vars, "In settings files").

The sink now also fills subject from data.file, so rows from guardrails and the formatters, whose data is {tool, file}, no longer land with an empty subject.

Fix

  • Launcher (harness-ops). A new async SessionStart hook, hooks/refresh-sink-launcher.sh, writes ${CLAUDE_PLUGIN_DATA}/hook-telemetry-sink.sh. It is async so it adds no startup wait; the hooks reference covers async SessionStart hooks (Run hooks in the background). That launcher runs this install's hooks/hook-telemetry-sink.sh. The data directory is ~/.claude/plugins/data/<id>/ and is "kept across plugin updates" (plugins reference, Environment variables). ${CLAUDE_PLUGIN_ROOT} changes on every update.
    • The launcher is rewritten only when its content changes, through a temp file and a rename, so a symlink at its path is replaced rather than followed.
    • It carries this install's session_event_log_dir and hook_events_max_bytes (or clears them), so a foreign producer's rows land in the same root as harness-ops' own.
    • When its install is gone, it deletes itself and exits 0.
  • Producer resolution (lib/hook-utils.sh, synced to all 20 carriers). hook::emit_telemetry maps @harness-ops to the launcher. Any other value starting with @ is reserved and skipped.
    • Lookup order: the caller's own data directory first, then harness-ops-* in name order. The first executable launcher wins.
    • The search root is the parent of CLAUDE_PLUGIN_DATA for a plugin hook. For a repository hook it is ${CLAUDE_CONFIG_DIR:-$HOME/.claude}/plugins/data; the CLAUDE_CONFIG_DIR row says "All settings, session history, and plugins are stored under this path" (env-vars).
    • The glob means a marketplace name is never hard-coded (-inline and -<marketplace> data dirs both exist in practice).
    • The lookup uses only builtins.
    • When no launcher exists, the event is skipped (fail-open).
  • context-guard mod. It resolves the same value through $.fs.list and $.fs.exists, since a mod has no CLAUDE_PLUGIN_DATA. Before this change it would have joined @harness-ops onto the project root and tried to run that path.
  • Sink. subject now comes from .data.subject // .data.file.
  • Docs. The contract README and CHANGELOG document the reserved value. The envelope stays at 1.1: this is not an envelope field, and the 1.0 sink-path rule set the same precedent. The harness-ops README makes @harness-ops the default, and its copy route now lists hook-telemetry-sink.sh, hook-utils.sh and session-log-lib.sh.
  • Changelog. Fragments for every carrier: minor for harness-ops and context-guard, patch for the rest.

Verification

  • plugins/harness-ops/hooks/refresh-sink-launcher.test.sh: 21 pass. Covers a silent first write, the launcher reaching the real sink, no rewrite when nothing changed, a rewrite after the root moves, options carried and a caller's options overridden, a stale launcher deleting itself, missing inputs, a symlink at the launcher path, and a directory at the launcher path.
  • plugins/harness-ops/hooks/hook-telemetry-sink.test.sh: 58 pass. The new {tool, file} case failed before the fix.
  • lib/hook-utils.test.sh: 702 pass. It adds 7 @harness-ops cases: the config dir, the $HOME fallback, a sibling data dir, own-dir precedence, no launcher, and an unknown @ value. The first 5 failed before the change.
  • claude plugin test plugins/context-guard: 328 pass, 0 fail, including 4 new cases. The 3 @harness-ops cases were run red before the change.
  • The other harness-ops hook suites all pass: audit-event-emitter 126, audit-session-id 27, harness-ops-paths 37, hook-failure-audit 193, session-event-log 183, session-retention 21, skill-usage-audit 35, measure-hook-log-budget (all passed), prerequisites-fleet (exit 0).
  • shellcheck is clean on every changed shell file.
  • scripts/validate-plugin-contracts.test.sh: 230 pass.
  • scripts/validate-plugins.sh --only "<all 20 carriers>": "All mod tests passed. All plugin manifests and the catalog validated."
  • scripts/sync-shared-copies.sh --check reports all 767 copies matching. --check-bump origin/main reports every carrier bumped. check-changelog-fragments.sh --check-required origin/main reports 20 of 20.
  • End-to-end: a temp HOME, then the real SessionStart writer, then the real guardrails hardcoded-path-check.sh with HOOK_TELEMETRY_SINK=@harness-ops. One row landed in <proj>/.observability/claude/sessions/<sid>.jsonl both as a repository hook (no CLAUDE_PLUGIN_DATA) and as a plugin hook (sibling CLAUDE_PLUGIN_DATA), with source: "envelope", hook_event_name: "PreToolUse" and subject: "src/a.txt" taken from data.file.

Related

  • melodic-software/medley#2307: the consumer case. medley can drop its forked sink and wire @harness-ops.

🤖 Generated with Claude Code

https://claude.ai/code/session_015XWwmcgXUX6XBUUZcBwQfQ

…rness-ops

A repository could reach harness-ops' reference sink only through a vendored
copy or a per-machine cache path, because settings env values are not shell
expanded. HOOK_TELEMETRY_SINK now takes the reserved value @harness-ops.

- harness-ops: a SessionStart hook (refresh-sink-launcher.sh) keeps
  ${CLAUDE_PLUGIN_DATA}/hook-telemetry-sink.sh, a launcher that execs the
  installed sink with this install's session_event_log_dir and
  hook_events_max_bytes; rewritten only on change, via rename, and it removes
  itself when its install is gone.
- lib/hook-utils.sh: hook::emit_telemetry resolves @harness-ops to the first
  launcher under the plugin data root (the caller's own data dir, then
  harness-ops-* in name order), found beside CLAUDE_PLUGIN_DATA or under
  CLAUDE_CONFIG_DIR / ~/.claude; skips when none, and skips any other @ value.
  Synced to every carrier.
- context-guard's mod resolves the same value through $.fs.list.
- The sink maps subject from data.subject, else data.file, so {tool, file}
  envelopes from guardrails and the formatters keep their subject.
- Contract README and CHANGELOG document the reserved value (envelope stays
  1.1); the harness-ops README makes @harness-ops the default and lists every
  file the copy route needs.

Closes #14227

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015XWwmcgXUX6XBUUZcBwQfQ
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-08T23:20:34.778023Z
We couldn’t complete this review. Comment @codex review to try again.
4d8184f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

.claude/hooks/hook-telemetry-sink.sh mirrors the harness-ops sink except for
its hook-utils.sh source repoint, and its drift test failed after the sink
gained the data.file subject fallback and the @harness-ops wiring header.
This repository's settings still wire the repo copy; switching them to
@harness-ops is a separate change.

Refs: #14227

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015XWwmcgXUX6XBUUZcBwQfQ
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 4fdf5aa4d6e865c3152f7df5ba3a048e1939423f: read 30 of 30 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #14279, incremental (a6e083d → 4fdf5aa)

No security issues found in the changes since the last review. This lane already reviewed an earlier head, so this pass reported only blocking findings (CRITICAL or IMPORTANT).

critical-findings: 0

Scope

I read all 30 files in the incremental list. files_read counts 11:

  • the 10 files outside the identical group;
  • plugins/animation/hooks/hook-utils.sh, the first path of the identical group.

The job counts the other 19 synced hook-utils.sh copies as read. I also looked at every copy's hunks and at each copy's path. Every copy lives in a plugin's hooks/ or skill scripts/ directory beside the plugin's other hook scripts, and none is at a path that changes what the content does.

What was checked

  • hook::harness_ops_sink_to (lib/hook-utils.sh and its copies):
    • Backslashes: they are folded to / in CLAUDE_PLUGIN_DATA and CLAUDE_CONFIG_DIR. Claude Code sets both values, or the user's own environment does, so they are not attacker input.
    • Second root: the config dir's plugins/data root is now searched after the sibling root. A file there can only be planted with the user's own write access, which already allows code execution as that user. No new trust boundary opens.
    • What can run: candidates are still only <root>/harness-ops-*/hook-telemetry-sink.sh files that pass -x. The chosen one is passed to printf -v, never evaluated. -nt is a builtin test, and no input reaches a shell string.
  • refresh-sink-launcher.sh: the new touch -- "$launcher" runs only when the content already matches. touch follows a symlink, but on that path the launcher was already checked with ! -L before it was read, so current is non-empty only for a regular file. The only effect is a changed mtime, so there is no write-through.
  • context-guard register.tsx: $.fs.exists became $.fs.stat with kind === 'file' and selection by mtimeMs. The launcher still runs as a single argv element through $.process.run, as before. The checks are no wider than the old exists test.
  • Tests, docs and changesets: prose and test cases only. No standing instruction is deleted or softened.
  • Trust controls: none touched. The diff does not touch trusted-actor lists, the trusted-trigger check, lane tokens, the kill switch, merge holds, workflows or plugin manifests.

Settled threads

Both threads (Windows path splitting, and name-order precedence) are marked lines changed since resolution: no, and the owner's replies state fixes that this head carries. I did not re-raise them.

Notes (not findings)

  • Newest launcher wins: a launcher with a future mtime would be picked. Only someone who can already write the user's plugin data directory can plant one, and that access already allows running code as the user.
  • No embedded instructions: the PR context file contains no text directed at this lane.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 2 inline finding(s) on a6e083d37353570eae74bb63fcc3294f3b24da72; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread lib/hook-utils.sh Resolved
Comment thread lib/hook-utils.sh Resolved
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 4fdf5aa4d6e865c3152f7df5ba3a048e1939423f: read 30 of 30 changed file(s); 0 finding(s), 0 as inline comments.

Code review (incremental, a6e083d → 4fdf5aa): harness-ops sink launcher precedence and Windows paths

Blocking-only review. This lane already reviewed an earlier head, so this pass reports blocking findings only. It found none.

Scope. I read every hunk of the 30 files in the incremental list. The 20 synced hook-utils.sh carriers are byte-identical copies. I counted their first path (plugins/animation/hooks/hook-utils.sh), and the job counts the rest. Each copy sits at its plugin's shared-library path, as before, so no copy's path changes what its content does. lib/hook-utils.sh makes the same change at its own line offsets, and I read it separately.

What changed, and how it holds up:

  • hook::harness_ops_sink_to (lib/hook-utils.sh)
    • Path separators. It now turns backslashes into / in both CLAUDE_PLUGIN_DATA and CLAUDE_CONFIG_DIR, so a Windows-native data path now reaches its parent directory.
    • Two search roots. It searches the plugin-data parent, then always the config dir's data root. When both roots are the same directory, a launcher is only compared with itself, and -nt returns false, so nothing changes.
    • Picking a launcher. It picks the newest executable launcher with -nt.
    • set -e. The &&/|| lists inside the loops do not trip set -e: a failing command inside an && list is exempt, and a compound command that fails only because of an exempt command does not exit.
    • Empty array. The roots array is never empty, so bash 3.2 under set -u is safe.
  • refresh-sink-launcher.sh now touches an unchanged launcher and exits. A symlink still fails the -f && ! -L read guard, so a symlink is still replaced rather than touched.
  • The mod in register.tsx now uses $.fs.stat and its mtimeMs. The same file already uses that API (register.tsx:358), and the test world in context-guard.test.ts stubs fs.stat with kind: 'file'.
  • Docs. The contract README, the CHANGELOG and both changelog fragments now describe the "most recently refreshed launcher wins" rule. They match the code.

Earlier findings. Both earlier inline findings are resolved. The owner replied, and both threads are marked lines changed since resolution: no, so I did not raise them again:

  • the Windows backslash path in CLAUDE_PLUGIN_DATA;
  • launcher choice by name order.

The incremental diff carries the fixes the owner described, and the new tests cover them:

  • lib/hook-utils.test.sh:
    • a backslash CLAUDE_PLUGIN_DATA still finds the sibling launcher;
    • a plugin hook falls through to the config dir;
    • the newest launcher wins, even over the caller's own.
  • refresh-sink-launcher.test.sh: the touch refreshes the mtime and keeps the inode.
  • context-guard.test.ts: the newer harness-ops-mkt launcher beats harness-ops-inline.

Closing issue #14227. Nothing in this increment changes how the change meets its three criteria (assessed at the previous head). The earlier Windows gap in criterion 2 is now addressed.

Not checked. I ran nothing. The test results, the red-before-the-fix runs and the sync and changelog checks are the author's claims and unverified here.

Other notes:

  • No plugin manifest changed, so there are no manifest lines.
  • Neither the PR context nor the settled threads contained instructions aimed at the review lane.
  • Spend: one touch per SessionStart, inside an existing async hook. It uses no API credit.

…west

Two review findings on hook::harness_ops_sink_to:

- A Windows plugin hook gets CLAUDE_PLUGIN_DATA with backslashes, so the
  `%/*` split never reached the data root and every @harness-ops event was
  dropped. Backslashes now fold to `/` first, and every producer also
  searches the CLAUDE_CONFIG_DIR / $HOME data root when its sibling search
  finds nothing.
- Name order picked a stale but present launcher (harness-ops-inline sorts
  before harness-ops-<marketplace>), splitting rows across two log roots.
  The refresher now touches its launcher at each SessionStart (content is
  still rewritten only on change), and resolution picks the newest launcher
  with -nt, for every producer including harness-ops' own hooks, so all of
  them send to one root. context-guard's mod does the same with
  $.fs.stat mtimeMs.

The contract README states the precedence.

Refs: #14227

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015XWwmcgXUX6XBUUZcBwQfQ
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 9, 2026
Merged via the queue into main with commit 1da6964 Nov 9, 2026
67 checks passed
@kyle-sexton
kyle-sexton deleted the feat/14227-harness-ops-sink-launcher branch November 9, 2026 02:03
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Route repository hook telemetry through the harness-ops sink without a copy

1 participant