Repository navigation
fix: restore skill text and eval files a prompt audit changed without eval evidence - #16379
Conversation
… eval evidence Revert the instruction cuts that #16330, #16331, #16333, #16334, #16336, #16337, #16339 and #16340 made to shipped skill and agent text with no before-and-after eval evidence: the 54 listed cuts plus 192 further hunks that removed or shortened instruction text, numbers, conditions, examples, reasons or dated records. Restore the eval case, prompt and grader files those pull requests changed to their earlier text. Case-only edits, stale facts corrected, upstream facts swapped for pointers and added text stay. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ba492bba3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The price-before-running grader's FAIL clause contradicted its own PASS clause and prompt (24 runs, 4.2 to 5 USD); x:read's GET-response detail belongs behind its existing pointer; monitor.md's dispatched-subagent exception matches the file's opening contract. Each was a correction or pointer swap, not a cut, so main's text stays. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The restored x-thread-unroll description carried the bare template https://threadreaderapp.com/thread/<id>.html, which the online link check extracts as https://threadreaderapp.com/thread/ and fails on a 404. Main's code span around it stays; the words are unchanged. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
claude-security-review reviewed Security review: incremental, b783a90 → 575a91a (PR #16379)No security issues found in this push. Because this lane had already reviewed an earlier head, this was a blocking-only review: only CRITICAL and IMPORTANT findings would have been reported. critical-findings: 0 Scope: I read every hunk of all 7 files in What I checked:
Spend: this push changes nothing that spends API credits or could create a charge. |
|
claude-review reviewed Code review: incremental, b783a90 → 575a91a (blocking only)I reviewed only the push since b783a90, and only for blocking findings, because this lane already reviewed an earlier head. The push changes 7 files, and I read every hunk in all 7. Earlier findings: the push resolves them. Each changed hunk puts back main's text where my earlier review showed the audit had made a correction or a pointer swap, not a cut:
The other files I flagged earlier are no longer in the pull request; it now has 200 changed files, down from 229. They include write-for-agents, audit-automation-gaps, firecrawl Not counted (advisory, dropped because this review is blocking only): the PR description still says 42 plugins get No new blocking findings in this push. Cost: the push only edits documentation text. It spends nothing and starts no billed resource. Verifier dropped 0 of 0 candidates (no candidates; no verifier subagent ran). |
The review round showed these restored passages were false or contradictory at head, so the audit's edits were corrections or pointer swaps, not cuts: stale line numbers and dates, a rollback tag given as the pin, size and gate counts, a read-count signal no script emits, a C:-only abort the trigger and code contradict, examples that break the rule beside them, house-style and C4-key facts, plugin-repo counts worded as the user's repository, and eval descriptions and graders that cite moved lines or contradict their own case. Main's text stays for each; fragments for plugins left with no change are dropped. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
No related issue: restores skill text and eval files a prompt audit changed without eval evidence
Summary
Ten prompt-audit pull requests cut shipped skill and agent text with no before-and-after eval evidence, and edited eval cases and graders without re-calibration.
docs/review/skills-and-agents.mdandplugins/playbooks/skills/skill-authoring/reference/skill-criteria.md("Cutting text") let a cut ship only with eval evidence of no loss. This puts the earlier text back where the audit's edit was a cut.Fix
plugins/playwright/skills/playwright/SKILL.md). 6 keep main's text because review showed the earlier text was false at head, so the audit's edit was a correction: the audit-automation-gaps checklist's inline ladder, bothLast updated:headers in the permission criteria files, the machine-healthC:-only abort, the promotion-evidence plan's line numbers, and the be-concise quotes from the ai-slop catalog.plugins/*/skills/andplugins/*/agents/that removed or shortened instruction text, or dropped a directive, number, condition, example, reason, emphasis label or dated record, restored to the pre-audit text.case.yamlcriteria and two user-interface graders). Eval edits that fixed a broken or contradictory case keep main's text: theprice-before-runninggrader, the testinge2e-mcp-server-no-uicase (its Skill grader needsSkillallowed), the multi-agent expected outcome, and the firecrawl and x descriptions whose earlier line numbers no longer match SKILL.md.patchfragment for each of the 39 plugins whose shipped files change, and anonefragment for mcp-tools and user-interface, whose only change is an eval file.Verification
scripts/check-changelog-fragments.sh --check: all fragments valid.scripts/sync-shared-copies.sh --check: all registered copies match.scripts/check-changed-skills.sh origin/main: 0 failed.scripts/check-changed-links-online.sh origin/main: every added or changed external link resolves; offlinelycheeover the changed markdown: 0 errors.check-affectedhook passed on each pushed head.Related
Restores text changed by #16328, #16330, #16331, #16332, #16333, #16334, #16336, #16337, #16339 and #16340.
Intentional-removal: the audit's replacement wording is removed on purpose; its earlier text returns until eval evidence shows no loss.
🤖 Generated with Claude Code