Skip to content

overengineering:audit: out-of-repo manifests bury repo findings in a single-repo audit; add a custody filter or a per-owner rollup #4596

Description

@kyle-sexton

The audit walks every settings scope and plugin hook manifest the harness merges, including user and machine scopes (skills/audit/SKILL.md:39-42; skills/audit/context/surface-walk.md:119-132). It writes one finding row per plugin manifest (surface-walk.md:96).

On melodic-software/.github, a 37-file repository, a run on 2026-09-27 produced 61 findings:

  • 24 were near-identical UNPROVEN rows for user-scope plugin hook manifests owned upstream (claude-code-plugins or third parties).
  • Further rows covered ~/.claude/settings.json and ~/.claude/CLAUDE.md.
  • Only 4 were actionable in the repository.

Walking out-of-repo scopes is intended ("Out-of-repo is never a reason to skip"). The gap is that no argument limits the walk to in-repo custody: the arguments are layer scope, unattended and a focus hint (SKILL.md:97-117), and the focus hint does not change the layer scope. The report template (context/report-template.md) also has no grouping by owner.

Merging rows in the artifact would change finding ids and orphan recorded judgments, because the item unit is part of finding identity (surface-walk.md:90). The fix belongs in the report.

Proposed fix, in order of preference:

  1. In the report only, collapse findings whose custody is out-of-repo into one summary row per owner (count, verdict mix, pointer to the artifact rows). Leave artifact rows and ids unchanged.
  2. Add a custody-scope argument (for example custody=repo) that records the out-of-repo layers as not walked in scope, so the merge rules carry their prior findings forward instead of closing them.

Acceptance: an audit of a small repository shows its in-repo findings first, with user-scope plugin manifests taking at most one row per owner in the report.

Related: a separate issue proposes a repo-only default scope for the overengineering entry in the playbooks repo-sweep hygiene catalog.

Found in: overengineering 0.4.13.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S5b57FoPJkJMe1BDwhExiG

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Sep 27, 2026
  2. kyle-sexton commented on Sep 27, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during triage.

    Triage: verified, briefed (decision-defaulted)

    Category: enhancement (Feature). Priority: priority: medium (assessed default; blocks no other work).

    Verification. On main (overengineering 0.4.13), the audit skill's discovery walks every settings scope the harness merges, user and machine scopes included, and states that out-of-repo is never a reason to skip. The agent-hooks layer pins its item unit to one container per plugin/extension hooks manifest, and that unit is part of finding identity. The arguments are layer scope, unattended, and a free-text focus hint that explicitly does not change layer scope. The report template has no grouping by owner or custody. The reported symptom (out-of-repo manifest rows outnumbering in-repo findings on a small repo) follows directly from these rules. Not a duplicate: #4597 covers the playbooks repo-sweep catalog entry's default scope, a separate surface and decision.

    Brief

    Goal. In a single-repo audit, in-repo findings are the first thing a reader sees, and out-of-repo findings (user/machine-scope settings and plugin hook manifests owned upstream) do not bury them.

    Behavior (recommended, option 1 of the issue).

    • The rendered report groups findings by custody: in-repo findings first, in full.
    • Findings whose custody is out-of-repo render as one summary row per owner (the owning plugin/marketplace or settings scope), carrying the count of findings, the verdict mix, and a pointer to the underlying rows in the findings artifact.
    • The findings artifact is unchanged: same rows, same finding ids, same item unit per layer, same scope semantics. Only the report view changes, so no recorded operator judgment is orphaned and the delta/realign lanes that read the artifact are unaffected.

    Acceptance.

    • An audit of a small repository renders its in-repo findings before any out-of-repo content.
    • Each out-of-repo owner takes at most one row in the report.
    • Finding ids and artifact rows for the same inputs are identical before and after the change.
    • The skill's existing evals still pass; one eval or fixture covers the per-owner rollup.

    Out of scope. Changing the walk itself, the item unit, or finding identity. A custody-scope argument (option 2) is not part of this item.

  3. kyle-sexton commented on Sep 27, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during triage.

    Decision defaulted: option 1, a report-only per-owner rollup of out-of-repo findings, leaving the findings artifact and finding ids unchanged — veto before merge.

    The alternative, option 2 (a custody=repo argument that records out-of-repo layers as not walked in scope), stays available as a later additive change; it is reversible and does not conflict with option 1. Option 1 alone meets the stated acceptance criterion and is the reporter's first preference.

  4. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Sep 27, 2026
  5. kyle-sexton commented on Sep 27, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during work-loop.

    Human-gated by the admission gate. The work-loop lane (instance melo-lap-001-wsl-2) reads this item (work-class: scoped, type Feature) as C3, feature-shaped. It adds a custody filter or a per-owner rollup to overengineering:audit. Feature-shaped C3 is human-gated by operator policy.

    Decision needed: admit this for autonomous dispatch (and confirm the triage brief's defaulted option), or keep it for attended work.

  6. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Sep 27, 2026
  7. self-assigned this
    on Sep 27, 2026
  8. kyle-sexton commented on Sep 27, 2026

    @kyle-sexton
    MemberAuthor
    No description provided.
  9. kyle-sexton commented on Sep 27, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during attend-queue.

    Operator decision (2026-09-27): hold until #4597 (default in-repo scope, now agent-ready) merges, then re-check whether a filter/rollup is still needed here. Stay parked.

  10. removed their assignment
    on Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.status: readyTriaged, unblocked, and fully specified; eligible to pick up.

    Fields

    No fields configured for Feature.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions