Repository navigation
feat(playbooks): add reference-independence and config-before-generator skill rules - #16612
Conversation
…or skill rules skill-authoring's criteria gain two rules: - Reference files: a reference works without the skill's steps. Another skill or a person loads it through the skill's invocation or a documented action, never by path, and acts on it without running the step sequence, so it names its own inputs. - Body: a shipped skill generates a repository-specific skill (as /app-control:map does) only where repository configuration cannot express the difference; when config can carry it, keep one shipped skill and read the config. Each rule has an eval case under plugins/playbooks/evals. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
…grader The config-before-generated-skill grader required an explicit config entry and failed rewrites that read the linter from the repository's own linter config on every run, which the rule also allows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
claude-review reviewed Code review: no findingsThis PR adds two rules to
It also adds one eval case per rule and a What I checked:
The eval scores and validator results in the PR description are the author's claims; I did not run them. Spend: the two new eval cases add plugin-eval runs on prepaid API credit (20 turns and 600 s max per run). This is not a hold. Verifier dropped 0 of 0 candidates. |
|
claude-security-review reviewed Security review: no security issues foundI read the whole diff: all 12 files on the read list. The two copies listed in What changed
Lenses checked
Spend: the new eval cases use prepaid model usage when evals run (two cases, up to 20 turns and 600 seconds each). That is not a hold. critical-findings: 0 |
No related issue: two authoring rules decided for the skill-authoring criteria, with no tracking issue.
Summary
playbooks:skill-authoring's criteria (reference/skill-criteria.md) had no rule that a reference file must work outside the skill's step sequence, and no rule for when a shipped skill may generate a repository-specific skill instead of reading repository config.Fix
Two short rules, each with its reason, in the sections that already hold the neighbouring rules:
/docs-hygiene:audit-encapsulationchecks) and acts on it without running the steps. So a reference names its own inputs and never leans on a value an earlier step set. No existing statement covered this; the nearest bullet ("Each pointer says what the file holds and when to read it") is about pointers, so the rule sits directly after it./app-control:mapgenerates a project's control skill) only where repository configuration cannot express the difference; when config can carry it, keep one shipped skill and read the config. Placed after "Layer on a skill, never restate it", the other rule against copies.Two eval cases, one per rule, and a
minorchangelog fragment for playbooks.Verification
Both new cases ran through
claude plugin eval(2 runs per arm, with-plugin and no-plugin baseline, haiku judge). The plugin was copied outside the repository first, because the CLI refuses a plugin whose repository has many linked worktrees.skill-authoring-reference-works-without-stepsskill-authoring-config-before-generated-skillreference-works-without-steps: the behavior grader passed in both arms. The base model already removes an obvious step dependency, so the Δ comes from the criteria-read grader. The case guards the rule against regression; it does not show the rule changes behavior.
config-before-generated-skill: the first run's grader required an explicit config entry, and it failed the with-plugin runs, which read the linter from the repository's own linter config on every run. That also satisfies the rule, so the grader now accepts either form. On the re-run, the behavior grader passed 2/2 with the plugin and 1/2 without it.
validate-cases.py plugins/playbooks/evals: no FAIL; the new cases add only the standard "judge samples need calibration" WARN every llm-graded case here carries.check-skill.sh plugins/playbooks/skills: skill-authoring PASS, 0 errors, 3 warnings, all pre-existing (SKILL.md is unchanged).plugins/playbooks/skills/skill-authoring/scripts/update.test.sh: 32/32 passed.Related
None beyond the rules above: no other issue or PR.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC