Skip to content

Epic: native-overlap tooling fix and route-to-compose sweep #4047

Closed
9 of 9 issues completed
Closed

Description

@kyle-sexton

Brief

TLDR

  • Repair the native-overlap tooling so it reports honestly on Claude Code 2.1.263: the inventory extractor's bundled-skill lane, per-lane integrity, the reverse-parity blind spot, and the seeded-pairs drift.
  • Move the marketplace posture from route-only to compose-where-sensible: a skill runs the native surface as a named step when it resolves and layers its own value around it.
  • Add a per-row integration field (route, wrap, suggest) to the native-surfaces store, separate from verdict, and a suggest grammar to the native-references convention.
  • Degrade gracefully: when a native surface does not resolve, skip the step, report it with the gating axes, offer the enable path, never re-implement the native job.
  • Deliver as one parent issue with per-unit sub-issues: the tooling fix first, then claude-ops, then every other plugin holding a store row, one unit in flight at a time.

Goal

A run of /claude-ops:audit-native-overlap on the current Claude Code build tells the truth lane by lane instead of collapsing to broken, and the skills this marketplace ships stop competing with the native surfaces they overlap: where a bundled skill does a strict subset of a skill's job, the skill composes it as a step and adds its own value (deeper inventory, reconciliation, its own agents); where a built-in command does related work a skill cannot invoke, the skill tells the user it exists at the moment that matters; and in every session where the native surface is gated, disabled, or absent, the skill still completes its own part and says what it skipped and why.

Constraints

  • A skill never asserts that a native surface is present, absent, enabled, or unavailable. The native-references convention's presence-gate rule holds for every new phrase and body section.
  • A skill never invokes a target it cannot identify. Bundled skills: name in the listing plus an advisory description check, skip with a warning on mismatch. Plugin skills (any marketplace, including melodic-software): the namespaced plugin:skill form, plus marketplace-qualified provenance from claude plugin list when the CLI resolves. Built-in commands: never invoked.
  • Whether a session is unattended is declared by the caller, never sniffed. An unattended run records a built-in suggestion in its output instead of asking.
  • Verdicts stay human-gated. No component is edited before its store row carries a verdict and an integration value.
  • The sweep contract stands: one plugin is one unit, closed only when its PR merges green, never two units in flight.
  • Every touched description must fit the 1,536-character per-entry cap after baking; trimming existing text to make room is allowed and gated by /skill-quality:check.
  • Baked text is self-contained: no phrase or Boundary section cites a file outside its own plugin.
  • Plain-bullet acceptance criteria (acceptance_criteria_format resolved to free-text from the default; the repo declares no convention home).

Acceptance criteria

  • inventory.py --binary-only on Claude Code 2.1.263 reports integrity ok or degraded with a non-empty bundled-skill lane, its eval suite passes, and validated_against moves to 2.1.263.
  • overlap.py detect on an inventory with exactly one broken lane exits 3, writes the candidates file, states per-lane floors, and marks only that lane's rows as not re-derivable.
  • overlap.py self-check flags a frontmatter description that names a bundled surface behind a presence condition without a gate token.
  • Every store row carries integration with one of route, wrap, suggest; the self-check rejects any other value, rejects wrap on a built-in command row, and rejects wrap or suggest on a session-provided row.
  • The generated view renders the integration column and --check stays in sync.
  • The two human-added audit-skill-visibility rows appear in reference/canonical-pairs.json, and a consumer-repo detect proposes them.
  • The native-references convention documents three grammars (route phrase, wrap Boundary section, suggest body sentence) with the class table stating which grammar each provenance class may use.
  • A wrapped skill, run where the native surface resolves, invokes it as a named step and reports the native result alongside its own.
  • A wrapped skill, run where the native surface does not resolve, completes its own part and reports the skipped step naming the four gating axes and the enable path.
  • While a bundled skill is present under skillOverrides: name-only, the wrapping skill treats it as not resolving.
  • A skill with a suggest row surfaces the suggestion at the start of the run when the built-in command covers everything the skill does, and at the end when coverage is partial, phrased conditionally.
  • Every touched skill passes /skill-quality:check, and each sweep unit's plugin takes its version bump and CHANGELOG entry.
  • The parent issue quotes the affected store rows, and each sub-issue names its unit and its predecessor.

Captured assumptions

  • The two coverage cases above (native absent, name-only override) are the author's proposal; the user approved the Brief as a whole without ruling on them separately. Revisit if a sweep unit finds a third trigger or state the wrapped skill must handle.
  • Wrap candidates on the current store are the bundled-skill rows: doctor (three rows), simplify (two), run, design (two), and the plugin-backed security-review. code-review against the CI lane is expected to stay route because a CI lane cannot type into a session. Each row's final integration value is still a human verdict recorded in the store. Revisit if a row's verdict moves to superseded.
  • Built-in command rows (export, skill-doctor) and the design access command take suggest. Revisit if Claude Code exposes a built-in command through the Skill tool.
  • The skill-doctor row keeps its gated marker on the docs basis (v2.1.252+, absent when feature-flag fetching is skipped); the extractor's gated: false reading is a heuristic blind to feature-flag gates. Revisit when the extractor learns to read that gate.
  • design is two surfaces sharing one name: the canvas bundled skill (the store row) and a gated built-in access command. The row's evidence must name both. Revisit if either is renamed.
  • Replace (our skill retired in favor of the native one) is expressed by the existing superseded verdict, so integration needs no replace value. Revisit if a row reaches superseded and still needs a runtime relationship.
  • The extractor fix targets the 2.1.263 bundle layout, where the readable export map no longer carries registerBundledSkill:()=>…; the new registration shape is an implementation finding for the plan. Revisit on the next CLI release that moves validated_against again.
  • Session-provided rows (morning) stay defer with integration: route withheld until an in-session capture protocol exists.

Scope note, 2026-09-10, added at planning after the fresh-context review: three statements about defer rows disagreed (the assumption above says route is withheld for morning, the acceptance criterion says every row carries integration, and the plan's verdict pass proposes route). The acceptance criterion is the stronger, approved statement, so every row carries integration, morning included, with route as the only value a defer row may take. The assumption's "withheld" is superseded by this note.

Scope note, 2026-09-10, added at planning after the devil's-advocate pass: the wrap-candidate assumption above rests on model-invocability that the 2.1.263 binary contradicts. Each bundled-skill registration carries an invocation-control field, and doctor, design, design-sync, batch, debug, and run-skill-generator are registered with model invocation disabled, which the official skills reference defines as "Claude cannot invoke" and this repository's invocation-mode convention restates as "cannot be invoked by any other skill". simplify and run are registered model-invocable. So the three doctor rows and the two design rows cannot take wrap on any host; their available integrations are route and suggest. This changes which rows the sweep can compose and is the user's decision at the policy unit's verdict pass, not resolved here.

Out-of-scope

  • Re-implementing any native job inside a marketplace skill as a fallback.
  • Retiring any skill under superseded in this effort.
  • Wrapping or suggesting session-provided skills.
  • The improvement items beyond the four broken ones: a recheck subcommand for trigger evaluation, a marker-basis field, name-collision keying in detect, an apply-time per-entry cap precheck, durable report output from a bare run, and a reverify helper. These are filed as follow-up scope below, not built here.

Deferred questions

  • none; every registered question closed answered.

Affected store rows

Every row of the native-surfaces store at the time of filing. Integration reads pending until the policy unit's human verdict pass writes it.

Native surface Class Component Verdict Integration
code-review (markers: none) bundled-skill review:code-review (skill) complementary pending
simplify (markers: none) bundled-skill code-tidying:tidy (skill) complementary pending
simplify (markers: none) bundled-skill code-tidying:batch-simplify (skill) complementary pending
security-review (markers: none) plugin-backed-builtin review:security-review (skill) complementary pending
run (markers: none) bundled-skill testing:run-e2e (skill) complementary pending
doctor (markers: gated) bundled-skill claude-ops:audit-install-state (skill) complementary pending
doctor (markers: gated) bundled-skill claude-ops:audit-performance (skill) complementary pending
doctor (markers: gated) bundled-skill claude-ops:audit-skill-visibility (skill) complementary pending
skill-doctor (markers: gated) builtin-command claude-ops:audit-skill-visibility (skill) complementary pending
morning (markers: none) session-skill claude-ops:morning-brief (skill) defer pending
export (markers: none) builtin-command session-flow:clean-stop (skill) complementary pending
playground (markers: none) marketplace-plugin visualization:visualize (skill) complementary pending
playground (markers: none) marketplace-plugin prototype:explore-directions (skill) complementary pending
design (markers: gated) bundled-skill visualization:visualize (skill) complementary pending
design (markers: gated) bundled-skill prototype:explore-directions (skill) complementary pending
design-sync (markers: hidden, gated) bundled-skill visualization:visualize (skill) defer pending

Two rows are added by the policy unit: export against session-flow:handoff and export against session-flow:retro, both suggest, because each body already carries a suggestion and every baked line must trace to a row.

Follow-up scope

Filed here for later items, not built under this container:

  • A recheck subcommand for the overlap engine that evaluates recheck triggers against a changelog and the upstream plugins repository, so trigger evaluation stops being a manual session act.
  • A marker-basis field on store rows recording which substrate set each marker.
  • Name-collision keying in detect (one native name resolving in two lanes).
  • Durable report output from a bare /claude-ops:audit-native-overlap run.
  • A reverify helper that stamps a row's date plus evidence without hand-editing JSON.
  • The registrar-export advisory in the inventory extractor is blind to the ESM export form; the tooling fix teaches it that form, and any further registration shape found later is its own item.
  • Phantom constant names from a global constant map (a same-named identifier bound to an unrelated string in another chunk); the tooling fix scopes resolution per chunk, and any residual case is its own item.

Search before create

Searched open and closed issues for native-overlap, audit-native-overlap, native-surfaces, and inventory.py 2.1.263 on 2026-09-10. No open issue carries this scope; the nearest matches were a closed item pinning the skill-doctor row's upstream source and the code-metrics follow-ups container, neither of which overlaps.

Execution shape: per-item PRs

Activity

  1. added theissue type on Sep 10, 2026
  2. kyle-sexton commented on Sep 11, 2026

    @kyle-sexton
    MemberAuthor

    Round-4 policy decisions for #4049 were resolved on 2026-09-11 and validated by two independent research agents against the official Claude Code references and the 2.1.263 binary. Consequences for this container:

    Corrections to the Brief's captured assumptions (recorded in docs/topics/audit-native-overlap/PLAN.md, third scope note): the name design carries two bundled registrations on 2.1.263, and the canvas skill both design rows describe is model-invocable, so those rows do not carry model-invocation-disabled and wrap is admissible for them (the accepted value is route on job merits, revisitable in units 7 and 8). The assumption that no built-in command is exposed through the Skill tool has fired: the official skills reference exposes /init and /security-review; the export and skill-doctor rows still take suggest because those commands are non-prompt command types.

    Unit 1 (#4048) gained a requirement: a same-name bundled collision rule so disable_model_invocation is keyed per registration. Unit 2 (#4049) stays work-class: structural and needs-human; the verdict table in its body is the C4 plan approval.

    Follow-up scope, added to this container:

    • /verify has no store row although it is the closest bundled analogue to testing:run-e2e and verification:confirm. Candidate pairs: verify x testing:run-e2e, verify x verification:confirm. It is model-disabled by a flag-driven field, so suggest at most.
    • A Native step identity check (name plus description) cannot see behaviour drift under a correct identity, such as simplify running single-pass without the Agent tool. Candidate grammar addition: a post-step assertion on what the surface reported doing.
    • Seeded-vs-observed name-collision keying in detect stays here; the bundled-vs-bundled case moved into claude-ops/inventory: recover the bundled-skill lane on 2.1.263 and report integrity per lane #4048.

    Generated by Claude Code

  3. 1 remaining item

  4. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Reopened. This comes from the 2026-09-29 audit of the Cursor agent run.

    Found. #5058 said "Refs #4047. ... It does not close #4047", but the negated closing reference still registered (github-actions warned on that PR), and #4047 closed 2026-09-28T08:33:46Z as a side effect of the merge. The epic's acceptance is not met: sweep unit #4051 is still OPEN. Its PR #4825 merged 2026-09-29 as Refs: #4051, with the live positive and negative simplify transcripts still outstanding and a stated deviation on unattended awaiting the operator.

    Open list: #4051 only. #4048, #4049, #4050 and #4052 to #4056 are closed; this comment does not re-judge them (their findings belong to the session-flow, testing and decisions-docs fix groups).

    Next. The owner closes the epic deliberately after #4051 is done. Nothing was merged by this run. Fix PR for the claude-ops findings: #5300 (Refs only).

  5. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Addition to the reopen comment above, from the 2026-09-29 audit of the Cursor agent run.

    Correction. The open list is #4051 and #4052, not #4051 only. #4052 (testing:run-e2e Native step) was reopened 2026-09-29T05:43Z by the testing fix group after cursor[bot] closed it before its blocker #4049 merged. #4048, #4049, #4050 and #4053 to #4056 are closed; this comment does not re-judge them.

    Container close ritual for this epic has not run. As of 2026-09-29 none of the three parts holds:

    The owner closes the epic deliberately, after those three hold.

    Store and skill half: fixed on main. code-tidying:batch-simplify has ## Native step: simplify (bundled skill) and both simplify store rows are baked, at a89707b83 (#4825, Refs: #4051). What #4051 still lacks is the two quoted live transcript runs (a positive run where simplify resolves, a negative run on a host with disableBundledSkills). That is operator work; nobody has faked it.

    Owner question (no option is implemented). #4051 says the Native step still runs under unattended. docs/conventions/native-references/README.md item 7 of the Native-step grammar says a caller-declared unattended "records instead of asks and never invokes a mutating surface", and #4825 records that both review lanes flagged the issue's version. #4825 follows the convention, so under unattended the step never runs.

    Deciding this unblocks the last wording in #4051's acceptance and its transcript runs.

  6. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Owner decision (2026-09-29): option (a). Keep the convention's rule: under unattended the batch-simplify Native step is skipped and simplify is never invoked (native-references README item 7). #4051's text is to match, and its live transcript proof covers the non-unattended path only.

    Next: #4051's Agent Brief already states the skip-under-unattended rule, so no wording edit remains. The epic stays open with needs-human. The owner runs the two live transcripts on #4051 (simplify resolving, and a host with disableBundledSkills), #4052 finishes, then the owner closes the epic after the close-out review and a closing comment linking the shipping PRs.

  7. kyle-sexton commented on Oct 22, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during attend-queue.

    Close-out check against the owner's intent (2026-10-22), at main f17e0de0da. The epic stays open. The owner closes it.

    The owner's intent is that native Claude Code surfaces are pointed to and composed (additive), and that native skills the model cannot invoke are surfaced to the person in interactive runs.

    Sub-issues and shipping PRs

    All nine sub-issues are closed.

    Unit Shipping PR(s)
    #4048 inventory #4092
    #4049 integration axis #5058
    #4050 harness-ops (was claude-ops) #4827, evidence #5302
    #4051 code-tidying #4825; live transcripts on the issue, owner-closed 2026-10-01
    #4052 testing #4828, #5265; live transcripts on the issue
    #4053 review #4806
    #4054 visualization #4810
    #4055 prototype #4810
    #4056 session-flow #4826

    Acceptance criteria at HEAD

    Permalinks below are at f17e0de0da.

    Criterion State Evidence
    Inventory reports per-lane integrity with a non-empty bundled-skill lane Met; the validated build has since moved validated_against is 2.1.288 (inventory.py:67). On 2.1.308, --binary-only reports degraded: 17 bundled skills, and the tools, plugins and workflows lanes broken, each named per lane. That is the honest per-lane report the criterion asked for. The drift itself belongs to the changelog lane and #6888.
    detect exits 3 on a broken lane and marks only that lane's rows Met On the 2.1.308 inventory, detect exited 3 and printed "lane(s) bundled_workflows, builtin_tools, builtin_plugins broken ... marked re_derivable: false" (overlap.py:1584)
    Self-check flags a description that has a presence condition but no gate token Met overlap.py:183-195; convention README:210
    Every row carries integration, and the class rules are enforced Met All 73 rows carry integration. Enum at overlap.py:91, class checks at overlap.py:797-848. self-check exits 0 (degraded on 2 version advisories).
    Generated view renders integration, and --check stays in sync Met native-surfaces.md:18; generate --check exits 0. CI runs both through validate-plugins.sh:128, 137.
    The two audit-skill-visibility rows are in canonical-pairs.json Met canonical-pairs.json:42, 47
    The convention documents the three grammars and the class table Met README:246-326
    A wrapped skill invokes the native surface as a named step, or skips and reports why Met Native steps at batch-simplify:10-51 and run-e2e:11. Positive and negative live transcripts are on #4051 and #4052. For #4051's item 3, the owner accepted a single-axis line (2026-10-01).
    A bundled skill listed under name-only is treated as not resolving Replaced by owner decision On 2026-09-11 the owner kept the downgraded rule: a name listed without a description is invoked, with the warning "identity confirmed by name alone" (README:285-286). The decision is recorded only in the spec that #5719 deleted: git show 209209449b^:docs/specs/native-overlap-route-to-compose.md, lines 365 and 404.
    A suggest row surfaces the suggestion at the start of the run (full coverage) or at the end (partial coverage) Partly met 11 of 32 suggest rows use the suggest-sentence shape, and the self-check traces them. The other 21 offer the surface in a **Routing.** paragraph that the self-check does not trace. Gap issue 1 below.
    Interactive runs only: unattended declared by the caller Partly met 9 of 29 wrapping or suggesting skills declare unattended, which audit-native-overlap SKILL.md:327-328 requires. visualize and explore-directions do not handle an unattended run at all. Gap issue 2 below.
    /skill-quality:check per unit, version bump and CHANGELOG per unit Met per unit Recorded on each closed sub-issue; not re-run for this comment.
    Parent quotes the affected rows; each sub-issue names its unit Met The Brief's "Affected store rows" table above

    Gap issues filed from this check

    1. fix(native-references): write the 21 unchecked suggest offers as suggest sentences so the self-check traces them #9103: write the 21 untraced suggest offers as suggest sentences, so the self-check enforces them.
    2. fix(native-references): declare unattended on every suggesting skill and record the design suggestion in unattended runs #9104: declare unattended on the 20 suggesting skills that lack it, and record the design suggestion in unattended runs of visualize and explore-directions.
    3. harness-ops/audit-native-overlap: rule on store rows for the model-disabled native surfaces with no row on Claude Code 2.1.308 #9105 (needs-human): rule on store rows for the model-disabled native surfaces that have no row on 2.1.308 (run-skill-generator, and the detect candidates rename, brief, resume, stop, agents, plus the model-invocable claude-code-docs).

    Close ritual (from the 2026-09-29 comment)

    • Every sub-item closed: yes.
    • Close-out review passed: none is recorded. This comment is a criterion-by-criterion check, not an independent review.
    • Closing comment linking the shipping PRs: the table above can serve as one once the owner accepts it.

    Follow-up scope with no issue

    No issue exists for these items from the Brief's "Follow-up scope" and the 2026-09-11 comment:

    • A recheck subcommand.
    • A marker-basis field.
    • Name-collision keying in detect.
    • Durable report output from a bare run.
    • A reverify helper.
    • A post-step behaviour assertion for wraps.

    The /verify row and the ESM-export and phantom-constant extractor items have landed.

    Recommendation: let the remaining items close with the epic and file none. None has recorded a stumble since 2026-09-11, and the per-lane integrity report plus the changelog lane cover the drift they were meant to catch. Basis: judgment, from the search of open and closed issues on 2026-10-22. If you want any of them kept, name it and it will be filed on its own before the epic closes.

  8. kyle-sexton commented on Oct 24, 2026

    @kyle-sexton
    MemberAuthor

    Closed by the owner's HITL session under 2C/Q87 (2026-10-24), with a second agent's independent agreement. All 9 sub-issues closed; gaps carried by #9103, #9104, #9105, and the six unfiled follow-ups moved to #9105. To reopen: reopen this issue and say what is still missing.

  9. added
    spec-containerSpec container: sub-issues are its work items (/work-items:decompose and /work-items:ship).
    and removed on Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.spec-containerSpec container: sub-issues are its work items (/work-items:decompose and /work-items:ship).

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions