Skip to content

effort: high is pinned in eleven agent definitions across three plugins, removing the operator's cheapest cost lever #4253

Closed

Description

@kyle-sexton

Effort outranks model tiering. Per https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence (fetched 2026-09-19, re-fetched raw and CONFIRMED by a fresh-context verifier): "in these internal measurements, a multi-model configuration that looked cheaper than the default single model cost more than that same model at lower effort", and "On DeepWideSearch, low also matched an orchestrator with a Claude Sonnet 5 worker at 29% lower cost: lowering effort beat an architecture change." Anthropic names low for "Simpler tasks that need the best speed and lowest costs, such as subagents" (https://platform.claude.com/docs/en/build-with-claude/effort, fetched 2026-09-19).

Per https://code.claude.com/docs/en/sub-agents.md (rung-1 curl, fetched 2026-09-19, 115,522 bytes, line 307): "Effort level when this subagent is active. Overrides the session effort level." No per-invocation effort parameter exists, so a frontmatter pin leaves an operator no supported way to lower worker effort.

Current state, from grep -rn '^effort:' plugins/*/agents/*.md:

  • plugins/implementation/agents/implementer.md:6
  • plugins/implementation/agents/phase-verifier.md:6
  • plugins/discovery/agents/explorer.md:8
  • plugins/discovery/agents/researcher.md:8
  • plugins/discovery/agents/intent-tracer.md:8
  • plugins/review/agents/code-reviewer.md:6
  • plugins/review/agents/architecture-guardian.md:6
  • plugins/review/agents/ci-log-auditor.md:6
  • plugins/review/agents/doc-drift-detector.md:6
  • plugins/review/agents/ecosystem-specialist.md:6
  • plugins/review/agents/security-reviewer.md:6
  • plugins/planning: Step 3 dispatches a generic read-only sub-agent (skills/plan/SKILL.md:149) with no plugin-owned definition, so it inherits session effort unalterably

The implement-dispatch packet (implementation-implement-dispatch/.../audit-notes.md F5, MEDIUM) records the rationale: it guards against an orchestrator lowering effort for its own bookkeeping. Its correction file evidence-1.md C4 leaves a premise open: "Every worker ran Opus at session effort" while the frontmatter pins high, so either the session was already at high or the pin did not take. That is unverified.

Cheapest fix: keep effort: high where source edits justify it (implementer, security-reviewer, architecture-guardian); drop or lower it for binary-criteria checking (phase-verifier, doc-drift-detector, ci-log-auditor). Fuller: expose the binding as a per-plugin userConfig key, and have planning ship agents/plan-reviewer.md with bounded effort.

Needs-decision: lowering effort trades capability; Anthropic documents an executor at low effort that stops detecting it is stuck, collapsing the advisor consult rate.

Verification: resolve C4 first. /tasks names a subagent's effort level when the definition sets it (sub-agents.md line 370, requires Claude Code v2.1.242 or later). Re-measure tokens per phase on one run at the new setting.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F7GFS5autRMSaea6kSoXzx

Activity

  1. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Decision packet. This comes from the 2026-09-29 audit of the Cursor agent run. The issue is reopened and needs an owner decision. status: needs-decision and work-class: structural are kept.

    1. Why reopened

    A status: needs-decision item was closed by PR #4974, merged by the Cursor agent with no review, choosing the option opposite to the issue's "Cheapest fix". The owner authorized fixing audit findings on 2026-09-29. No effort decision is made here and no agent file is changed.

    The linked PR (#5268, Refs #4253) only amends the record in docs/plugin-philosophy.md so it is true:

    • It names CLAUDE_CODE_EFFORT_LEVEL as the coarse all-lanes override. Upstream: frontmatter effort "overrides the session level but not the environment variable" (model-config.md); sub-agents.md says "Overrides the session effort level".
    • It states that no per-lane or per-plugin lever exists.
    • It fixes the count: 13 high pins and 2 medium (re-counted on origin/main with grep -rE '^effort:' plugins/*/agents).
    • It drops the "Option A" label and the forward-binding "Do not unpark a funded sweep" sentence.

    2. The question

    Keep all effort: high pins, lower some, or add a lever.

    3. Options

    • A. Keep all 13 pins (status quo). The record now states the operator cost honestly.
    • B. Lower or drop effort: high on the three binary-criteria checkers the issue names (phase-verifier, doc-drift-detector, ci-log-auditor). Keep it on the source-edit and decision lanes (implementer, security-reviewer, architecture-guardian, and so on).
    • C. Expose a per-plugin userConfig effort key that actually reaches the worker, and give planning a bounded-effort reviewer (plan-reviewer already pins medium), so the operator gets a per-lane lever without a source edit.

    4. Owner steps for the open verification

    This is the issue's "resolve C4 first". The agent did not run it; it needs an interactive session.

    1. In a Claude Code session at v2.1.242 or later, lower the session effort below high.
    2. Dispatch a pinned agent (for example review:code-reviewer).
    3. Run /tasks and read the subagent's effort level. Expected high if the pin takes effect over the session level.
    4. Repeat with CLAUDE_CODE_EFFORT_LEVEL set in the environment and confirm the environment value wins.
    5. Optional: re-measure tokens per phase on one run at any new setting.

    Upstream text already says the pin overrides the session level, so step 3 confirms rather than discovers.

    5. Recommendation

    B in its narrow form, at medium not low, after step 3 confirms the pin takes effect.

    Basis: the issue's own "Cheapest fix" and the two Anthropic pages it cites (optimizing-for-cost-and-intelligence, effort). medium over low is judgment, from the issue's own caveat that an executor at low effort stops detecting it is stuck.

    6. What the decision unblocks

    The record in docs/plugin-philosophy.md can then state the decision, and a sweep can be scoped or closed. The issue's alternatives (funded sweep, userConfig effort key, the C4 check) stay tracked here; no separate follow-up issue is filed.

  2. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Review-plugin specifics for the decision packet above. This comes from the 2026-09-29 audit of the Cursor agent run. The issue is already reopened and status: needs-decision stays; no option is implemented and no agent frontmatter is changed.

    Facts from the review plugin.

    • The effort: high pin is on plugins/review/agents/{code-reviewer,architecture-guardian,ci-log-auditor,doc-drift-detector,ecosystem-specialist,security-reviewer}.md, in frontmatter. The packet above counts 13 high pins across plugins; the review plugin holds six of them.
    • PR docs: keep effort:high agent pins; record operator cost (#4253) #4974 recorded "Option A: keep all eleven pins" in docs/plugin-philosophy.md with no per-agent assessment. "Option A" is not defined in this issue. The record correction is PR docs: fix factual and wording defects in top-level docs and the native-surface store #5268 (core-docs); the doc edit stays with core-docs.
    • The doc says CLAUDE_CODE_EFFORT_LEVEL is a coarse global override and no per-lane lever exists.
    • The issue's verification step (resolve C4: /tasks shows a subagent's effort on v2.1.242+, then re-measure tokens per phase) was skipped. Owner steps are in section 4 of the packet above.

    Decision state. The #4974 record is provisional. It is not an owner decision. The issue's own counterweight applies to any lowering: an executor at low effort stops detecting that it is stuck.

    Options and recommendation are unchanged from the packet above (A keep all, B lower the binary-criteria checkers, C add a per-plugin userConfig key). The review plugin's candidates under B are doc-drift-detector and ci-log-auditor; security-reviewer and architecture-guardian stay high.

    What this unblocks. The lane edits, the userConfig key, and the correction to docs/plugin-philosophy.md.

    needs-human is added if absent.

  3. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Sep 29, 2026
  4. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Correction to the decision packet and the review addendum above: both count 13 effort: high pins, but origin/main has 14 high and 1 medium (discovery:research-verifier went from medium to high in #5301, merged 2026-09-29), so read "Option A: keep all 13 pins" as all 14; the recount is git grep -h '^effort: high' -- 'plugins/*/agents/*.md' | wc -l, and #5268 now records 14 and 1.

  5. kyle-sexton commented on Sep 29, 2026

    @kyle-sexton
    MemberAuthor

    Owner decision (2026-09-29): Option B, narrow: lower effort from high to medium (not low) on the three binary-criteria checkers (phase-verifier, doc-drift-detector, ci-log-auditor), after confirming the pin takes effect. security-reviewer and architecture-guardian stay high. Owner's words: "Accept recommendation: B, narrow, at medium not low, after confirming the pin takes effect".

    Next: the confirmation is the owner step in section 4 of the decision packet (an interactive session on v2.1.242+; /tasks shows the pinned subagent's effort, then repeat with CLAUDE_CODE_EFFORT_LEVEL set). needs-human stays until that is done. Then the three frontmatter edits and the docs/plugin-philosophy.md record can proceed.

  6. kyle-sexton commented on Sep 30, 2026

    @kyle-sexton
    MemberAuthor

    Resolution (2026-09-29, owner-approved consensus)

    Goal. Lower the pinned effort on three binary-criteria checker agents from high to medium, so an operator gets a cheaper default for that work. Keep high on the two agents where a missed defect costs the most. This is the owner's "Option B narrow" decision, at medium, not low: the issue notes an executor at low effort stops detecting that it is stuck, so medium is the compromise.

    Steps.

    1. Recount on the branch: git grep -h '^effort: high' -- 'plugins/*/agents/*.md' | wc -l.
    2. Open a draft PR. Change effort: high to effort: medium in these three files only: plugins/implementation/agents/phase-verifier.md, plugins/review/agents/doc-drift-detector.md, plugins/review/agents/ci-log-auditor.md. security-reviewer and architecture-guardian stay at high.
    3. Update the decision record in docs/plugin-philosophy.md (the record correction in docs: fix factual and wording defects in top-level docs and the native-surface store #5268 already merged). State Option B narrow at medium, name the three lowered agents and the two kept at high, and change the recorded count from 14 high / 1 medium to 11 high / 4 medium, using the recount from step 1.
    4. Title the PR in Conventional Commits form and write the body to the PR body contract (closing-keyword line, Summary, Fix, Verification, Related). Use Refs #4253, not Closes, until the confirmation in step 5 is done.
    5. Confirm the pin takes effect with a read-only local probe on this WSL host (claude 2.1.285, above the 2.1.242 floor). Run headless claude -p at a session effort below high, dispatch a pinned agent such as review:code-reviewer, and read the effective effort from the transcript or telemetry. Repeat with CLAUDE_CODE_EFFORT_LEVEL set. /tasks is interactive-only, so if the headless probe cannot show the subagent's effort, leave that check to the owner (lower session effort, dispatch the agent, run /tasks, repeat with the variable).
    6. Keep the PR in draft until the probe or the owner confirms the pin takes effect. Then flip it with /source-control:pull-request ready.

    Acceptance criteria.

    • Exactly three agent files change (effort: high to effort: medium); security-reviewer and architecture-guardian still say high.
    • docs/plugin-philosophy.md names Option B narrow, the three lowered agents, the two kept, and the count 11 high / 4 medium, matching the branch recount.
    • PR title is Conventional Commits and the body meets the contract; it uses Refs #4253 until confirmation is done.
    • The confirmation result (probe output, or the owner's /tasks check) is recorded in the PR body under Verification.
    • Rollback is a one-line frontmatter revert per file; keep the phase-verifier change easy to revert if its verdict quality drops.
    • effort: high is pinned in eleven agent definitions across three plugins, removing the operator's cheapest cost lever #4253 is closed as completed only after the PR merges. Only the owner approves the ready flip and the merge.
  7. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Sep 30, 2026
  8. kyle-sexton commented on Sep 30, 2026

    @kyle-sexton
    MemberAuthor

    Every acceptance criterion in the owner-approved Resolution comment is met on origin/main by PR #5538 (merge commit c79aa29). The owner (kyle-sexton) merged it at 2026-09-30T03:55Z.

    What I checked on origin/main:

    • git grep '^effort: high' over plugins/*/agents gives 11, and the same count for medium gives 4.
    • phase-verifier, ci-log-auditor and doc-drift-detector are set to effort: medium.
    • security-reviewer and architecture-guardian are still effort: high.
    • docs/plugin-philosophy.md, in "Pinned effort: high agents", states "Eleven named agents pin effort: high ... four pin effort: medium". It names the owner's decision, Option B narrow at medium and not low, the three lowered agents and the two kept at high.
    • The PR title follows Conventional Commits, and the body meets the PR body contract and says Refs #4253.
    • The Verification section records the headless probe. With --effort low, the subagent still ran at effort high, so the pin takes effect. With CLAUDE_CODE_EFFORT_LEVEL=low set, the environment variable overrides the pin. Resolution step 5 accepts this probe in place of the owner's /tasks check. The PR itself notes that it was a headless run and that the interactive /tasks check was not done.
    • Rollback is a one-line frontmatter revert per file.

    The issue now carries agent-ready and has neither needs-human nor status: needs-decision. The last criterion is to close #4253 as completed after the PR merges, and it has merged. No open PR references 4253. No new work remains.

  9. kyle-sexton commented on Oct 3, 2026

    @kyle-sexton
    MemberAuthor

    For the record, after close: phase-verifier's effort pin went back to high, and two probes show how a pin behaves under Workflow dispatch.

    Current state on main. #5538 set phase-verifier, ci-log-auditor and doc-drift-detector to medium under this issue's Option B narrow decision. #5885 (merged 2026-10-02) then set phase-verifier to effort: high and moved ci-log-auditor and doc-drift-detector to opus at high. docs/plugin-philosophy.md now records 10 high and 6 medium pins.

    Why phase-verifier belongs at high. These are the reasons from the spending-your-effort interview (Q9), which reached the same level as #5885:

    • The high row in model-config's Choose an effort level table says "Work where verification matters or edge cases are likely". Spending your effort puts verification work at high too. Both are more specific than the model's general default of medium.
    • The reason fix(review,implementation): lower effort pins on three binary-criteria checkers to medium #5538 gave for medium, "the extra effort bought cost without changing the verdict", is not backed by any measurement. The doctrine's own recheck trigger ("a checker pinned medium misses a defect its high pin caught") has never been tested.
    • In the post, the low-effort runs failed because they skipped the checks a verifier exists to run.
    • Users can still run it lower when they want to: a Workflow call's effort option and CLAUDE_CODE_EFFORT_LEVEL both override the pin.

    Probe results (Claude Code 2.1.284 and 2.1.285, local Workflow runs, transcripts read for perTurnEffort and model):

    1. A Workflow agent() option overrides a named agent's frontmatter. agent({agentType: 'planning:plan-reviewer', model: 'sonnet', effort: 'xhigh'}) ran on claude-sonnet-5-5 at xhigh, even though the agent's frontmatter says opus/medium (run wf_1a471686-8a2, 2.1.284, one probe).
    2. A Workflow agent() call that names a pinned agent and passes no effort keeps the pin. phase-verifier ran at high from a medium session (run wf_5196f26b-e1f, 2.1.285).

    One more result from the same session: a skill whose frontmatter sets effort: high passes that level to the subagents it dispatches. The test ran from a low session, and a control skill with no effort line stayed at low (2.1.285).

    Together with the headless probe in #5538 (the --effort low flag left the pin in force, while CLAUDE_CODE_EFFORT_LEVEL=low overrode it), these cover how the pin interacts with the session level, the environment variable, and Workflow options. The Agent tool has no per-call effort parameter, so through the Agent tool the pin decides the level.

    No action requested; the issue stays closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions