Skip to content

chore: apply prompt-audit fixes to songwriting through tdd plugins - #16340

Merged
kyle-sexton merged 6 commits into
mainfrom
chore/prompt-audit-songwriting-to-tdd
Nov 15, 2026
Merged

kyle-sexton merged 6 commits into
mainfrom
chore/prompt-audit-songwriting-to-tdd

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

No related issue: prompt-audit findings from an owner-requested full-read audit

Summary

A full-read prompt audit (Anthropic's prompt-audit guide) found instruction text written for older models, stale against the repository, or contradicted by another file. This PR applies the verified findings for the songwriting, source-control, spelling, sql-server and tdd plugins (group 9 of 10, 124 files). Every edit was confirmed by a second agent before it was written.

The change is instruction text only, except for five code files whose edits are confined to text a model or user reads:

  • plugins/source-control/scripts/worktree-create.sh: usage text and one comment, emphasis lower-cased.
  • plugins/source-control/scripts/worktree-root-doctor.sh: note/warn message strings and comments, emphasis lower-cased except the three strings its test asserts.
  • plugins/source-control/skills/babysit-prs/scripts/babysit_merge.py and babysit_resolve_thread.py: module docstrings only, emphasis lower-cased.
  • plugins/sql-server/src/tools.mjs: the description strings of query, list_objects, describe_object and write now state the return shape, the default row cap and request timeout, and the failure modes. Each claim matches src/executor.mjs (DEFAULT_LIMITS, the returned objects, the write verdict) and src/statement.mjs (the refusals).

Fix

  • Capital-letter emphasis markers lower-cased (ADR 0098).
  • Past-tense revision history, incident IDs and migration-relative phrasing replaced with the rule as it stands.
  • Stale paths, commands and claims corrected to what the repository holds; restated upstream behavior replaced with pointers.
  • Contradictions between files settled toward the file that owns the rule. One example: the babysit docs said the resolve wrappers refuse a severity-flagged thread whoever asks; babysit_resolve_thread.py applies that screen only under --autonomous and --independent-resolver, so the docs now say the agent carries the escalation in the other resolve modes. The rule itself (escalate a security/P1 thread in every tier and mode) is unchanged.
  • tdd: the Child Test trigger and the integration-test share of the pyramid now match the Beck and Khorikov references the skill summarizes.
  • Lint fixes the audit edits needed: a double blank line, a line that started with # (MD018), labelled to labeled (typos), the severity sentence in babysit-prs/SKILL.md reflowed to stay under the 500-line cap, and worktree-root-convention.md removed from scripts/as-of-recheck-baseline.txt because its as-of date now carries a recheck trigger, as the check asked.
  • A patch changelog fragment for each of the five plugins.

Reverted audit hunks:

  • plugins/songwriting/agents/object-writer.md (the fix(songwriting): object-writer returns several pivot sentences where it is asked for one #6031 observed-stumble evidence), plugins/source-control/skills/babysit-prs/reference/loop.md ("Quality over speed") and plugins/source-control/skills/pull-request/reference/prep.md (default parallel verification agents): cuts or behavior rewrites in skill and agent text that docs/review/skills-and-agents.md allows only with before-and-after eval evidence, which this PR does not carry (Codex review finding).
  • Three message strings in plugins/source-control/scripts/worktree-root-doctor.sh keep their capitalized WITH, AFTER/ABOVE and INSIDE: worktree-root-doctor.test.sh cases 38, 43 and 47 assert them.

The pr-refine/fix-docs lane added 6b364c1, which rewrites figurative phrasing in 26 songwriting reference files into literal wording.

Verification

  • git apply --3way of the group patch: every hunk applied cleanly on current main; no file dropped.
  • plugins/source-control/scripts/worktree-root-doctor.test.sh: 0 failures after the message reverts.
  • scripts/sync-shared-copies.sh --check: all 904 registered copies match.
  • scripts/check-changelog-fragments.sh --check and --check-required origin/main: pass for all five plugins.
  • scripts/check-affected.sh --tests: 17 checks passed after the lint fixes; scripts/check-changed-skills.sh: 19 skills, 0 failed. Suites past the local 120 s budget are left to CI.
  • node --test plugins/sql-server/test/server.test.mjs plugins/sql-server/test/defaults.test.mjs: 67 passed, 0 failed.

Related

  • Other groups of the same audit land as separate pull requests.

🤖 Generated with Claude Code

kyle-sexton and others added 3 commits November 15, 2026 09:03
Apply the verified prompt-audit findings for the songwriting,
source-control, spelling, sql-server and tdd plugins: lower-case
capital-letter emphasis (ADR 0098), drop history narration, correct
stale paths and claims, and settle contradictions between files toward
the owning file. The sql-server tool descriptions now state each tool's
return shape, default limits and failure modes.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Collapse a double blank line, keep an issue list from starting a line
with a hash, use the American spelling the typos gate wants, and drop
worktree-root-convention.md from the as-of recheck baseline now that
its as-of date carries a recheck trigger.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Reflow the severity-screen sentence onto two lines so the skill stays
below the 500-line hard cap check-skill enforces.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 15, 2026 14:09
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T14:12:29.931743Z da7b8e6 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da7b8e61ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43722988348

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/apply-patch
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 6b364c1
Run 43722988348

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

Restore three skill and agent passages the audit cut or reworded
without the before-and-after eval evidence docs/review/skills-and-agents.md
requires: the object-writer return rule's observed-stumble evidence
(#6031), babysit-prs' "Quality over speed" note, and prep's default
dispatch of parallel verification agents.

Restore the capitalized words in three worktree-root-doctor.sh messages
that worktree-root-doctor.test.sh asserts (cases 38, 43 and 47).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 23c5e2070adbab8abfe6f9e89ba6155b08a1463a: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, blocking only): PR #16340 at 23c5e207

I found no security issues in this push.

What I read. I read every hunk in .claude-lane/incremental.diff for the one file this push changed since ba3ead3c: plugins/songwriting/context/pat-pattison/research/phrasing.md. This lane already reviewed an earlier head of this pull request (blocking only: yes), so this review reports only CRITICAL and IMPORTANT findings.

What changed. Lines 612–617 of phrasing.md now name where Pat Pattison's one sourced "body language" sentence lives: the patpattison.com "The Art of Phrasing" quote in lyric-melodic-roadmaps.md. The old text said no source had been found. The edit is research prose for a songwriting context file. It governs no tool, permission, write, credential, posting or publishing action. It removes no guardrail; it narrows an attribution rule ("quote no other line as his").

Other checks.

  • Trusted paths and trust controls: the file matches no TRUSTED PATHS glob. Nothing touches the trusted-trigger check, the lane token, the kill switch, a merge hold or a plugin manifest, so there are no manifest lines.
  • Embedded text: I found no instructions aimed at this lane in the PR context or prior-findings files.
  • Spend: none.

critical-findings: 0

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 23c5e2070adbab8abfe6f9e89ba6155b08a1463a: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

claude-review: incremental review of ba3ead3c..23c5e207, blocking findings only. I read the one hunk of the one changed file and found no blocking findings.

This push changes the "Body language" section note in plugins/songwriting/context/pat-pattison/research/phrasing.md:613-617. It used to say no source for a Pat quotation on "body language" had been found. It now says the framing must not be attributed to a book, points to Pat's one sourced quote from the patpattison.com "The Art of Phrasing" column in lyric-melodic-roadmaps.md, and says to quote no other line as his.

I checked the pointer by reading the files:

  • lyric-melodic-roadmaps.md:251-254 holds the quote ("Phrasing has the power to create emotion. It's the body language of your…"), credited to "Pat Pattison, patpattison.com, "The Art of Phrasing"".
  • The comment at lyric-melodic-roadmaps.md:247-248 and the note at line 35 agree that the quote comes from that column.
  • The rest of the plugin says the same: phrasing.md:571-574, book-references.md:205, beyond-books.md:151 and response-filter.md:243.

So the new note no longer contradicts the roadmaps file.

Notes:

  • I did not run any tests or checks. Everything above comes from reading the files.
  • No plugin manifest changed in this push.
  • I found no instructions embedded in the PR context, settled-threads or prior-findings files.
  • The prior-findings file says blocking only: yes, so this pass reports only blocking findings. The one settled thread (object-writer.md) has no changed lines since it was resolved and is outside this push.

Verifier dropped 0 of 0 candidates.

@github-actions

Copy link
Copy Markdown
Contributor

The pr-verify verifier failed head ba3ead3c5a9c3cb630619bc38a98019dc76c5932. What it wrote, as data:

What it wrote (53 lines)
Summary: Chunk 1 of 3, fail: The spec is the trusted PR #16340 body (spec_kind pull-request, no
  linked issue): prompt-audit fixes to the songwriting, source-control, spelling, sql-server and tdd
  plugins. I read chunk 1 whole: all 20 parts, each in one Read. These are the five changelog
  fragments, songwriting/agents, the songwriting context, research and template files, and
  co-write/SKILL.md. I also read chunk-2/part-001 whole to check the skill actions these files name.
  One edit in phrasing.md adds a false claim, and that blocks the pass.

What chunk 1 delivers in full:
- A `patch` changelog fragment for each of the five plugins.
- The reverted object-writer.md hunk. Only DIFFERENT and IS are lower-cased, and the #6031 evidence
  sentence is kept.
- The figurative-phrasing rewrites in the songwriting reference files from the fix-docs lane.

What chunk 1 delivers in part, for the songwriting files only:
- Capital-letter emphasis lower-cased.
- Past-tense revision history and incident notes restated as present-tense rules.
- Stale commands and paths corrected. The datamuse.sh calls now use the `${CLAUDE_PLUGIN_ROOT}` path
  that line-emission-gate.md already uses. The `/variations`, `/audit`, `/rhyme-generation` and
  `/mosaic` hand-offs now name `/songwriting:diagnose variations|audit` and `/songwriting:rhyme
  [mosaic]`, and chunk-2/part-001 confirms those skill actions exist. The SKILL.md layout pointers
  now go to artifact-persistence.md.
- Contradictions between files settled. Examples: the co-writing solo rules, the
  five-compositional-elements numbering, the lonely/only identity error in the rhyme-generation
  example, and title-game deferring to hook.md.
- Lint fixes: the `relabelled` sentence is gone.

What chunk 1 does not hold, because it sits in other chunks: the source-control, spelling,
  sql-server tools.mjs and tdd edits, the babysit severity-screen docs, and the
  as-of-recheck-baseline change.

Checks:
- Every upstream job concluded success on the judged head, including Lint Repo, Check Skills and
  Check Plugins
  (https://github.com/melodic-software/claude-code-plugins/actions/runs/43724681747/job/132844356827).
- The capture is blocked (subscription token behind proxy), which is no evidence either way.
- No binary files, no takeover-set files, and no text trying to steer the reviewer.

Not a defect, but unconfirmed: response-filter.md now cites the rhyme-type scale to Chapter 4, p.
  27. The unchanged line-edit-rubric.md still cites Chapter 5's chapter-opening scale, and
  rhyme-generation.md cites Chapter 9, p.110. The book may print the scale in more than one place,
  so I could not establish an error.

The blocking defect: the phrasing.md edit adds "or quote him on it; no source for such a quotation
  has been found". lyric-melodic-roadmaps.md quotes Pat's "body language of your song" sentence from
  patpattison.com "The Art of Phrasing". The edit therefore creates a new cross-file contradiction
  and repeats the demotion error that file warns against.
- plugins/songwriting/context/pat-pattison/research/phrasing.md:615: the added clause "or quote him
  on it; no source for such a quotation has been found" is false. lyric-melodic-roadmaps.md quotes
  Pat's "Phrasing has the power to create emotion. It's the body language of your song." from
  patpattison.com "The Art of Phrasing", and this file's own front-heavy/back-heavy section credits
  the body-language frame to Pat's column. Change the sentence to say the two unsourced quotations
  that once stood here are not Pat's, and that the one sourced "body language" quote is the column
  one cited in lyric-melodic-roadmaps.md. Or drop the added clause.

This pull request leaves the merge queue, and ci-status holds it out while it carries pr-verify-failed. A pr-verify pass removes the label, at a new commit or at this same head: once a trusted person edits the body, and only trusted people have edited it since the fail, this head is judged again, so a fail on the body or the issues it links needs only the body fixed, no new commit.

@github-actions github-actions Bot added the pr-verify-failed pr-verify failed this head; a pass at a new commit, or after a trusted body edit, removes it. label Nov 15, 2026
The audit's rewrite said no source exists for quoting Pat on body
language, but lyric-melodic-roadmaps.md quotes his patpattison.com
column "The Art of Phrasing". Point at that quote and keep the rule
against attributing the framing to a book.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the pr-verify-failed pr-verify failed this head; a pass at a new commit, or after a trusted body edit, removes it. label Nov 15, 2026
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 15, 2026
Merged via the queue into main with commit 7cced2d Nov 15, 2026
52 checks passed
@kyle-sexton
kyle-sexton deleted the chore/prompt-audit-songwriting-to-tdd branch November 15, 2026 15:21
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant