Skip to content

docs: re-apply prompt-audit history, provenance and stale-fact fixes - #16410

Merged
kyle-sexton merged 5 commits into
mainfrom
chore/prompt-audit-reapply
Nov 15, 2026
Merged

kyle-sexton merged 5 commits into
mainfrom
chore/prompt-audit-reapply

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

No related issue: follow-up to the prompt audit; re-applies rule-backed edits that #16379 restored

Summary

#16379 restored skill and agent text a prompt audit had changed without eval evidence. It also restored text the repository's own rules forbid: history and provenance notes, migration-relative wording ("now", "no longer", "as before") and as-of dates on plain pointers. Removing those is not a cut under plugins/playbooks/skills/skill-authoring/reference/skill-criteria.md ("History and provenance") or .claude/rules/skill-bodies-state-current-rules.md. This puts those edits back and leaves every real cut restored.

Fix

  • 134 history, provenance, migration-wording and pointer as-of-date edits re-applied, by exact text match where the passage was unchanged and by hand where it had moved.
  • 9 emphasis edits: all-caps stress on ordinary words lower-cased, nothing else changed.
  • 4 stale facts corrected after checking each against the repository: a rule count and rule kinds in the ai-slop catalog, a recheck step naming an as-of date the pointer no longer carries, a "this repository" example in a shipped skill, and a contents link to a section that no longer exists.
  • Left restored, not in this change: edits that change behavior or drop a number still in use, eval case and grader text, wording a test pins by phrase or digest, and stale-fact findings the repository does not prove wrong.
  • A patch changelog fragment for each of the 34 plugins whose shipped files change.

Verification

Related

Follows #16379, which restored text changed by #16328, #16330, #16331, #16332, #16333, #16334, #16336, #16337, #16339 and #16340.

Intentional-removal: history and provenance notes, migration-relative wording and as-of dates on plain pointers are removed on purpose; the skill criteria forbid them in skill text.

🤖 Generated with Claude Code

kyle-sexton and others added 3 commits November 15, 2026 13:29
The restore pull request (#16379) put back skill and agent text a prompt
audit had changed without eval evidence. It also put back text the
repository's own rules forbid: history and provenance notes,
migration-relative wording and as-of dates on plain pointers. Those are
not cuts under the skill criteria's "History and provenance" section.

This re-applies those edits, lower-cases all-caps emphasis on ordinary
words where no test pins the wording, and corrects stale paths, counts
and cross-references confirmed against the repository. Edits that change
behavior, drop a number still in use, or touch eval judge text stay
restored.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 15, 2026 18:44
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T18:46:23.948661Z e0df149 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43748479921

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 2cc93a9254f1ba557b2dd84540dad546de23dff9: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, cc08476 → 2cc93a9): no security issues found

critical-findings: 0

I read the whole incremental diff. It touches one file: plugins/implementation/skills/implement-dispatch/reference/command-evidence.md. No trusted-path glob matches it.

What changed: one sentence in the "Basis" note at command-evidence.md#L23. It used to say that item 2 evidence needs a project's own hook. It now says that item 2 comes from a project's own hook or a transcript tool result.

Instruction-surface lens: this does not widen the evidence rule. Item 2 at L14-L16 already lists "a transcript tool result matched by its tool-use id", and that line is unchanged. The edit makes the Basis note agree with it. Item 3 is unchanged: a worker's prose, a DEVIATIONS.md line or the orchestrator's memory still never counts as evidence. The file says it is guidance only and gates no action, credential or untrusted input. I did not consult the protected-class list in docs/contracts/instruction-exception-register/README.md; I judged the edit by its consequence alone.

Other surfaces: the change touches no manifest, userConfig key, workflow, action, script, hook or trust control. It removes no named friction hold.

Spend: none.

@kyle-sexton kyle-sexton added the do-not-merge Hard merge gate: do not merge while applied. label Nov 15, 2026
@kyle-sexton

Copy link
Copy Markdown
Member Author

Hold (do-not-merge) added by the GitHub Support hub at Wayfinder's request. This pull request changes plugins/prototype/skills/explore-directions/reference/bundled-design.md. The before/after evals running for #16346 and #16321, and the pending one for #16348, read that file. A merge during those runs voids them, as #16336 and #16379 each did today.

The hold comes off when #16346, #16321 and #16348 have merged, or sooner if this pull request stops changing that one file. Its other 144 files raise no concern.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on e0df149caf544a2efae205b18382f7f3106590f9; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 2cc93a9254f1ba557b2dd84540dad546de23dff9: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Incremental review of cc08476a..2cc93a92: blocking only, no findings.

This review covered only the blocking class, because the prior-findings file says blocking only: yes.

The push changes one line, plugins/implementation/skills/implement-dispatch/reference/command-evidence.md:23. It applies the wording this lane suggested in the earlier thread (#16410 (comment)) word for word. The basis line now names both sources of item 2 that the list gives at lines 14-16: "a log a consuming project's hook appends per Bash call, or a transcript tool result matched by its tool-use id". It also matches line 27, which presents the hook as one option ("A project that wants item 2 as a hook"). An orchestrator or phase-verifier reading the basis line will no longer reject transcript tool-result evidence. That resolves the earlier finding, and the edit raises no new problem.

I had no shell and ran nothing. There is no closing issue, so there were no acceptance criteria to check. The incremental diff, prior-findings file and PR context held no embedded instructions aimed at this lane.

Verifier dropped 0 of 0 candidates.

@kyle-sexton

Copy link
Copy Markdown
Member Author

Commit cc08476 takes plugins/prototype/skills/explore-directions/reference/bundled-design.md out of this pull request: the file now matches main, and the prototype changelog fragment is removed with it. git diff origin/main -- plugins/prototype .changes/prototype on the head is empty, so this pull request no longer touches anything the evals for #16346, #16321 and #16348 read.

That meets the hold's second condition ("sooner if this pull request stops changing that one file"). The label is the hub's to remove; this session leaves it in place and does not merge while it is set.

…weep-all figures

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kyle-sexton kyle-sexton removed the do-not-merge Hard merge gate: do not merge while applied. label Nov 15, 2026
@kyle-sexton

Copy link
Copy Markdown
Member Author

Hold lifted by the GitHub Support hub: head 2cc93a9254 changes no file under plugins/prototype (142 files in the diff, read with the paginated files API), which meets the hold's second condition.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 15, 2026
Merged via the queue into main with commit 79fef96 Nov 15, 2026
118 of 122 checks passed
@kyle-sexton
kyle-sexton deleted the chore/prompt-audit-reapply branch November 15, 2026 19:22
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant