You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #6476. Rewrite wizard's skills to the skill criteria in one draft PR, following the per-plugin runbook (/playbooks:skill-authoring, reference/rewrite-a-plugin.md). The runbook and the rewrite protocol it cites decide what ships; this issue holds only what is specific to the plugin.
The plugin today
As of origin/main on 2026-10-24:
Skills: generate, unattended (model-invoked).
Outcome eval case directories under the plugin: 0. Probe files under probes/: 0.
Done when
/skill-quality:check plugins/wizard on the PR head reports no FAIL that the run on main does not.
/evals:validate passes on plugins/wizard's eval suite at the PR head.
Each model-invoked skill has a probe file: plugins/wizard/probes/generate.json, plugins/wizard/probes/unattended.json.
The PR body holds a per-skill decision table, description and body decided separately, with the before and after numbers behind each row.
The PR body names every skill or case left unmeasured, with the reason.
The plugin's version or changelog entry is in the PR, per the plugin's release mode.
Eval spend and review
Eval passes within prepaid API credit are pre-approved (owner decision Q46, 2026-10-24), at the --max-cost-usd 5 per-pass ceiling set on #6476 on 2026-10-18.
No person gates the PR: the owner's 2026-10-24 decision (C12) leaves only an intent mismatch during the build or a security issue at review as stops.
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Medium
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: none
Understood: rewrite the two model-invoked wizard skills (generate, unattended) to the skill criteria in one draft PR, following the per-plugin runbook. That includes adding probe files, running the skill-quality and eval checks, a per-skill decision table in the PR body, and a version or changelog entry. The scope and done-when items are clear. Eval spend is recorded as pre-approved within prepaid credit. No likely duplicates found in the open issues. This is part of #6476.
Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.
The work-resolve-issue lane stopped partway through this issue and needs a person.
Why, in the run's own words:
The issue requires the rewrite protocol in plugins/playbooks/skills/skill-authoring/reference/rewrite-a-plugin.md and skill-criteria.md 'Measurement', which ships only parts that were measured: a before/after /skill-quality:check run, eval cases written through /evals:design and checked with /evals:validate, probe sets checked with /skill-quality:check measure-invocation, paid /evals:plugin-eval passes for each version on each target model, and a /docs-hygiene:compress compare meaning diff. This CI lane loads only the work-items plugin, so none of those skills is available. Bash runs only the named wrappers, so it cannot run claude plugin eval or spend API credit. Rewriting without measurement would break the protocol's 'ship only parts that were measured' rule. Writing the probe files here would also skip the required checks and break the rule that keeps probes away from the rewriter. Nothing was changed. The issue needs a session that has the evals, skill-quality and docs-hygiene plugins plus eval API credit.
What the run says it did not do:
/skill-quality:check plugins/wizard before/after comparison: the skill-quality plugin is not loaded in this lane
/evals:validate on plugins/wizard eval suite and outcome cases via /evals:design: the evals plugin is not loaded and paid eval passes cannot run here
Probe files plugins/wizard/probes/generate.json and unattended.json: the protocol needs a fresh-agent probe set checked with /skill-quality:check measure-invocation, which is unavailable here
Per-skill decision table with before/after numbers: no measurement possible in this lane
Plugin release record (version or changelog fragment): nothing shipped, so none added
Parked for capacity, not for the owner. This rollout's remaining work is the measured eval passes of the batch-1 protocol, which the issue lane cannot run: it cannot start claude or claude plugin eval (see the lane's escalation on #6585, https://github.com/melodic-software/claude-code-plugins/actions/runs/40186335064). It waits on the plugin eval dispatch work in #9704, now a blocked-by edge, so admission holds it until #9704 closes.
Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So needs-human comes off, and the issue moves to host-run with status: confirmed, which keeps the issue lane from taking it when #9704 closes: #9704 adds an eval dispatch workflow, not eval capability for the issue lane.
Shipped: 6 outcome eval cases under plugins/wizard/evals/ (regex and tool_used graders) and frozen trigger probes probes/generate.json and probes/unattended.json.
No skill text changed. A concise hub-and-spoke rewrite of both skills was measured (40 passes), and every part reverted on the validity rule in skill-criteria.md Measurement step 4: 7 of 8 trigger passes were judged INVALID for sandbox denials the probes themselves trigger, and each body had one INVALID outcome sub-pass.
Activity
github-actions commentedon Oct 7, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Medium
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: none
Understood: rewrite the two model-invoked
wizardskills (generate,unattended) to the skill criteria in one draft PR, following the per-plugin runbook. That includes adding probe files, running the skill-quality and eval checks, a per-skill decision table in the PR body, and a version or changelog entry. The scope and done-when items are clear. Eval spend is recorded as pre-approved within prepaid credit. No likely duplicates found in the open issues. This is part of #6476.Automated triage, applied by the lanes App.
kyle-sexton commentedon Oct 24, 2026
Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.
github-actions commentedon Oct 25, 2026
The triage sweep dispatched intake triage for this issue (attempt 1 of 2).
melodic-automation-lanes commentedon Oct 25, 2026
The work-resolve-issue lane stopped partway through this issue and needs a person.
Why, in the run's own words:
What the run says it did not do:
Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40190522123
To retry, remove
needs-humanand addagent-readyagain.kyle-sexton commentedon Oct 27, 2026
This was generated by AI.
Parked for capacity, not for the owner. This rollout's remaining work is the measured eval passes of the batch-1 protocol, which the issue lane cannot run: it cannot start
claudeorclaude plugin eval(see the lane's escalation on #6585, https://github.com/melodic-software/claude-code-plugins/actions/runs/40186335064). It waits on the plugin eval dispatch work in #9704, now a blocked-by edge, so admission holds it until #9704 closes.Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So
needs-humancomes off, and the issue moves tohost-runwithstatus: confirmed, which keeps the issue lane from taking it when #9704 closes: #9704 adds an eval dispatch workflow, not eval capability for the issue lane.kyle-sexton commentedon Oct 31, 2026
kyle-sexton commentedon Oct 31, 2026
Result: landed in #11672.
plugins/wizard/evals/(regex andtool_usedgraders) and frozen trigger probesprobes/generate.jsonandprobes/unattended.json.skill-criteria.mdMeasurement step 4: 7 of 8 trigger passes were judged INVALID for sandbox denials the probes themselves trigger, and each body had one INVALID outcome sub-pass.