Repository navigation
feat(architecture): report scan defects on their own and route review-class problems to their skills - #16194
Conversation
…s own A design scan can reproduce a live bug while looking for shallow modules. Until now the defect stayed inside its refactor candidate and surfaced only if that candidate was picked. The deepening action now names each reproduced live defect in the reply, security defects first. A defect that is not a security one goes to /bugs:write when that plugin is installed, else a report in the reply. A security defect is stated to the user and kept out of any text that may become public, with a pointer to private reporting. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…wning skills A design scan also notices problems that are not about module shape, such as types that opt out of checking and discarded errors. The deepening action now keeps them as evidence on their candidate and names the owning skill in the reply, without findings of its own. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 337d6e9ba5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
claude-security-review reviewed Security review (incremental, blocking only): 2 manifest flags, no exploitable issueThis lane already reviewed an earlier head, so this review reports only CRITICAL and IMPORTANT findings. Manifest flag lines are reported at either severity. What I read. The incremental scope has 11 files, all in the new eval case The manifest (
The manifest has no Everything else in scope.
Not a finding: this change adds no workflow, token scope, trusted-actor entry, kill-switch path or merge hold. Spend. The new case adds eval runs (3 runs per arm, up to 40 turns, a 900 s timeout). That spends prepaid model credit only and is not a hold. Embedded text. The PR context holds no text aimed at this lane. Prior findings. The only earlier post is the previous summary, which reported no findings, so nothing needs to be restated. critical-findings: 0 |
|
claude-review reviewed Code review: PR #16194 (incremental, blocking only)No blocking findings. This lane already reviewed earlier heads of this PR, so this pass reports blocking findings only. ScopeThis push adds one new eval case,
What was checked
Settled threadsThe owner settled the thread on Closing issueThe PR closes no issue, so there are no acceptance criteria to check. Notes
Verifier dropped 0 of 0 candidates. |
Three cases reach the deepening Phase 2 reply from a stated Phase 1 result over a scaffolded repository: - deepening-defect-reported-on-its-own: a reproduced non-security bug gets its own report (steps, expected against actual, fix location) before the candidate question; a dead-code claim is not a defect. - deepening-security-defect-kept-private: a missing access check is named first as a security defect, routed to private reporting, and no gh issue or pr create or git commit runs (Bash granted). - deepening-review-class-names-owner: Any typing, a discarded exception and a hard-coded rate stay as candidate evidence, with the owning skill named or the problems said to be not assessed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ile it The report for a reproduced non-security defect appeared only in the reply, so a session that ended before the user asked for it to be filed left no bug record. The scan now calls /bugs:write --file, which saves the report and offers the tracker filing; nothing is filed without the user's answer. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The run loads neither code-metrics nor review, so the skill's correct reply is that the review-class problems were seen and not assessed. The case accepted naming the owning skill as an alternative; it now requires the not-assessed statement, and naming the would-be owners beside it still passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… path The three defect-report cases load only the architecture plugin, so they exercise the fallback report and never the route the scan takes when the bugs plugin is installed. This case loads a stand-in bugs plugin from its own directory, the only place the eval CLI loads a second plugin from, and grades that the scan calls /bugs:write with --file, names the defect before the candidate question, and runs no tracker write or commit. The stand-in carries the write skill's name and --file contract only, so what the real skill does with --file is not measured by this case. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
No related issue: two owner decisions on what a design scan does with problems that are not about module shape, small enough to land directly.
Summary
architecture:improvereproduces any live-bug claim a scan makes before it reaches the report, but said nothing about where the defect goes next. It stayed inside its refactor candidate, so it surfaced only if that candidate was picked, and nothing kept a security defect's details out of a public tracker. The scan also had no rule for code-review problems it notices along the way.Fix
/bugs:write --filewhen that plugin is installed, so the report is saved to a file and the user is offered the choice to file it; nothing is filed without the user's answer. Otherwise the reply carries the report (title, reproduction steps, expected against actual, fix location)./code-metrics:audit type-debtor/review:quality-gate code, each when installed; the scan adds no finding or badge for them.Verification
plugins/architecture/evals/, each 3 runs per arm on Claude Code 2.1.325:deepening-defect-reported-on-its-own,deepening-security-defect-kept-privateanddeepening-review-class-names-ownerpassed 3 of 3 with the plugin and 0 of 3 without it. These load only the architecture plugin, so the defect case exercises the fallback report.deepening-defect-goes-to-bugs-write(906c717): a stand-inbugsplugin inside the case directory puts/bugs:writein the skill list, because the eval CLI loads a second plugin only from inside the case directory. Run twice, 3 runs per arm: the with-plugin arm passed 6 of 6 (the scan called/bugs:writewith--file, named the defect before the candidate question, and ran no tracker write or commit). The no-plugin arm loads neither plugin, so its failure is built in; 3 of its 6 runs failed the graders and 3 did not start, on a sandbox error about the machine's Docker credential store. That leaves the run-validity verdict at INVALID for both runs, on the baseline arm only./bugs:writedoes with--file. Checked once outside the eval harness: a headless session with this branch'splugins/architectureandplugins/bugsboth loaded, on the same scaffolded repository, in auto permission mode. The scan reproduced the defect, called/bugs:write --file, the skill wrote a 52-line report withtype: bug-reportfrontmatter to the bugs plugin's data directory, the reply gave its path and the command that would file it, and nothing was filed or committed. Two earlier attempts inacceptEditsmode with no approval surface had that write denied (the data directory is outside the working directory); both replies then carried the report inline and said the file was not saved.check-affectedpassed in the pre-push hook on each pushed head.Related
N/A
🤖 Generated with Claude Code