Skip to content

chore(pr-pipeline): count max-turns issue-lane runs that still opened a PR before tuning the turn cap #9499

Closed

Description

@kyle-sexton

Problem

17 of 91 work-resolve-issue failures (2026-10-16 to 2026-10-23) ended class=max-turns, and one more ended with 'successful result after N turns, exceeding' yet still opened a PR (#8655 -> #9483). GitHub Support count, E21. The cap should not be tuned until we know how many of the 17 produced a PR anyway.

Done when

Each of the 17 runs is classified (PR opened / no PR), and the cap is kept or changed on that evidence, per #7901's sizing-stop reporting.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 23, 2026
  2. github-actions commented on Oct 23, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: read-only
    Basis: automated intake triage; the lanes App applied this class.

    Size: Small
    Complexity: Low
    Uncertainty: Knowable
    Flags: none
    Blocked by: none

    Decision recorded.
    The owner answered in the Refinement session (2026-10-25), deciding the open items below on the facts:

    go with whatever the facts/data/metrics + /discovery:research says and ensure we're aligned with SOLID, DRY/SSOT principles

    I should only ever be in the loop ONLY when necessary (intent/vision changed, something came up during implementation that impacts trajectory/alignment, OR a high security risk) - I've covered this all before

    [quote truncated; the full decision is at the link]

    #9499 (comment)

    The decision settles the open questions: this is a report-only task. It classifies the 17 max-turns work-resolve-issue runs as PR opened or no PR, counts #8655 -> #9483 separately, and recommends keeping or changing the cap, with no config change. If run logs can't be read unattended, a person supplies the run IDs and outcomes.

    Automated triage, applied by the lanes App.

  3. added and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 23, 2026
  4. melodic-automation-lanes commented on Oct 23, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. It is model text, to check, not instructions.

    Intended outcome: A one-time report classifying each of the 17 max-turns work-resolve-issue runs as PR opened or no PR, with a recommendation to keep or change the cap.

    Dependencies

    • #​7901: defines sizing-stop reporting of max-turns that the cap decision should follow

    Related work

    Open questions

    1. Which 17 runs are in scope, and how are they selected (run IDs or a query)?
      RECOMMENDED: Select work-resolve-issue runs from 2026-10-16 to 2026-10-23 whose failure class is max-turns, and list their run IDs in the report. Basis: judgment
    2. Is the deliverable only a report, or should the run also change the cap?
      RECOMMENDED: Report only; the owner decides any cap change in a follow-up, because the cap is derived from base-turns times a complexity multiplier. Basis: .github/actions/resolve-config/dimensions.mjs:74
    3. Can an unattended run read the Actions run logs and artifacts needed to classify PR opened vs no PR?
      RECOMMENDED: Confirm access; if not, a maintainer supplies the 17 run IDs with their PR outcomes. Basis: judgment
    4. What threshold of PR-opened share would justify keeping versus raising the cap?
      RECOMMENDED: Keep the cap if most of the 17 opened a PR; raise it only if most opened none and each was nearly finished. Basis: judgment
    5. Should the 'successful result after N turns, exceeding' case (#​8655 -> #​9483) count as PR opened?
      RECOMMENDED: Yes, count it as PR opened and report it separately from class=max-turns failures. Basis: 8253

    Sources read

    • /home/runner/work/_temp/refine-data/context.json: issue 9499, no comments, 17 of 91 failures hit max-turns, no run IDs listed
    • .github/actions/resolve-config/dimensions.mjs:74: max-turns is ceil(size base-turns * complexity multiplier)

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

  5. kyle-sexton commented on Oct 26, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. The owner answered in the Refinement session (2026-10-25), deciding the open items below on the facts:

    go with whatever the facts/data/metrics + /discovery:research says and ensure we're aligned with SOLID, DRY/SSOT principles

    I should only ever be in the loop ONLY when necessary (intent/vision changed, something came up during implementation that impacts trajectory/alignment, OR a high security risk) - I've covered this all before

    Question asked: Accept a report-only deliverable (classify the 17 runs), with any cap change decided by the owner in a follow-up?

    1. (intake Q1) Which 17 runs?
      Decided: work-resolve-issue runs from 2026-10-16 to 2026-10-23 with failure class max-turns, listed by run ID in the report. Basis: kept, judgment; body Problem.
    2. (intake Q2) Report only, or change the cap too?
      Decided: Report only. The cap is base-turns times a complexity multiplier, so a change is a config decision. Basis: kept; .github/actions/resolve-config/dimensions.mjs:74.
    3. (intake Q3) Can an unattended run read the runs' logs and PR outcomes?
      Decided: Try it; if not, a person supplies the 17 run IDs and outcomes. Basis: kept, judgment.
    4. (intake Q4) What PR-opened share keeps the cap?
      Decided: Keep it if most of the 17 opened a PR; raise it only if most opened none and each was nearly finished. Basis: kept, judgment. fix(pr-pipeline): explain turns above max-turns and report max-turns as a sizing stop #7901's sizing-stop reporting, which the body cites, has no trace on origin/main (no "sizing-stop" match under .github or docs), so the decision cannot lean on it yet.
    5. (intake Q5) Does feat(source-control): ship the pr_open_state default as ready #8655 -> feat(source-control)!: ship the pr_open_state default as ready #9483 count as PR opened?
      Decided: Yes, reported separately from the class=max-turns failures. Basis: kept; body Problem names it.

    Acceptance criteria:

    • Each of the 17 runs is classified (PR opened / no PR) in a report on the issue, with run IDs (body's Done when, first half).
    • The report recommends keeping or changing the cap on that evidence.

    Checks a person runs:

    • The owner keeps or changes the cap on that evidence (body's Done when, second half).
  6. melodic-automation-lanes commented on Oct 26, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Deliver a report on the issue that classifies each of the 17 class=max-turns work-resolve-issue runs (2026-10-16 to 2026-10-23) as PR opened or no PR, and recommends keeping or changing the turn cap, without changing any config.

    Acceptance criteria

    • A report comment on the issue lists every work-resolve-issue run from 2026-10-16 to 2026-10-23 that ended class=max-turns, by run ID, and states the total found (expected 17).
    • Each listed run is classified PR opened or no PR, with the PR number when one was opened.
    • The #​8655 -> #​9483 run, which succeeded over the turn cap and still opened a PR, is reported separately from the class=max-turns failures and counted as PR opened.
    • The report states the PR-opened count against the no-PR count.
    • The report recommends keeping the cap if most runs opened a PR, or raising it only if most opened none and each was nearly finished, and names which applies.
    • If the run logs or PR outcomes cannot be read unattended, the report says so and lists the run IDs or outcomes a person must supply instead of guessing.
    • No file is changed; the cap stays base-turns times the complexity multiplier at .github/actions/resolve-config/dimensions.mjs:74 until the owner decides.

    Dependencies

    • #​7901: sizing-stop reporting is cited by the body but is not on origin/main, so the recommendation rests on the run data alone.

    Related work

    Sources read

    • .github/actions/resolve-config/dimensions.mjs:74: max-turns is Math.ceil(size base-turns * complexity multiplier), so a cap change is a config decision
    • /home/runner/work/_temp/refine-data/context.json: owner's Decision recorded comment settles report-only, the keep/raise rule, and counting #​8655 -> #​9483 as PR opened

    Automated refinement by the lanes App. Intake triages the issue again with it.

  7. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 26, 2026
  8. added
    admission-deferredAdmitted but waiting for WIP room; the hourly work-unblock sweep labels it agent-ready.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    and removed
    admission-deferredAdmitted but waiting for WIP room; the hourly work-unblock sweep labels it agent-ready.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 29, 2026
  9. melodic-automation-lanes commented on Oct 29, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue lane stopped partway through this issue and needs a person.

    Why, in the run's own words:

    Report only, per the recorded decision. No file changed, so the lane has nothing to commit, and the owner decides the cap from this report.

    Scope: the 91 failed work-resolve-issue runs created 2026-10-16..2026-10-23 (the Actions list filtered to is:failure). Each run's class comes from its run-page annotations: class=max-turns, or 'exceeding the configured maximum of 75'. 18 runs are class=max-turns, not the 17 the issue expected. Every one had cap 75.

    PR opened (15). Each PR came from the capped run's own branch, work-resolve-issue/issue-<n>-<run>-1:

    • 39113363166 #7650, 85 turns: #8232 merged
    • 39106125028 #6008, 95: #8191 merged
    • 39106121444 #6888, 76: #8223 merged (Refs)
    • 39106103357 #7333, 76: #8168 merged
    • 39106064278 #8013, 79: #8190 merged
    • 39106048556 #8009, 76: #8163 merged (Refs)
    • 39105818532 #7440, 83: #8103 merged (Refs)
    • 39098409432 #7375, 119: #8051 merged
    • 39092901535 #7934, 125: #7949 merged
    • 39092042611 #6075, 88: #7940 merged
    • 39092038343 #6055, 89: #7942 merged
    • 39089680523 #7644, 140: #7896 merged
    • 39089676786 #7667, 94: #7923 merged
    • 39089661280 #7693, 78: #7897 merged
    • 39089639607 #7764, 157: #7899 closed unmerged, superseded by #7918

    No PR (3). Each run stopped for a person, and a later, fresh lane run's PR closed the issue:

    • 39106056280 #8011, 80: partial work left on its branch; later #9637
    • 39105897350 #6445, 83: commit failed with 'Failed to get branch reference: 403'; later #9994
    • 39090491415 #7898, 87: a sandbox-blocked sync script and test failures; later #8024

    Reported separately (an over-cap success): 40012699365 #8655, 79 turns, annotated 'counted as a success', opened #9483 (merged).

    Count: 15 PR opened vs 3 no PR; 16 vs 3 counting #8655.

    Recommendation: keep the cap (base-turns x complexity multiplier, .github/actions/resolve-config/dimensions.mjs:74). Most runs opened a PR, so the keep branch of the decided rule applies. On every run page whose annotation showed the SDK result (#7934, #6075, #6055, #7898, #7644, #7667, #7764), the result was a success over the cap. Those runs had finished their work, and the cap did not cut it short. Two of the three no-PR runs stopped on causes other than turns: a 403 and a sandbox block. The pages read do not show #8011's cause.

    What the run says it did not do:

    • Run logs need a GitHub sign-in, so each class comes from run-page annotations read through WebFetch summaries, not raw logs; a few long annotations were paraphrased past about 120 characters
    • 18 class=max-turns runs found against the 17 the issue expected; the source of the GitHub Support count (E21) was not available here to reconcile the one-run difference
    • The cap decision itself is the owner's, per the recorded decision; no config changed
    • Tidy skipped: the run changed no file
    • Review, build and tests skipped: there is no diff to review or build

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40831885491

    To retry, remove needs-human and add agent-ready again.

  10. kyle-sexton commented on Oct 29, 2026

    @kyle-sexton
    MemberAuthor

    Report delivered in the lane's comment above (run 40831885491: 18 max-turns runs, 15 still opened a PR). The read-only work is done; the turn cap is tuned from this data under the owner's standing direction.

  11. kyle-sexton commented on Oct 31, 2026

    @kyle-sexton
    MemberAuthor

    Cap kept: issue-dimensions base-turns x complexity (docs/conventions/pr-pipeline.yaml:777,781, dimensions.mjs:74) and the standard profile's 75 (:682), by the rule decided on 2026-10-26, "Keep it if most of the 17 opened a PR": 15 of 18 class=max-turns runs opened a PR, 16 of 19 with #8655 (run 40831885491). No file changes. This corrects the close comment, whose "tuned" was wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.status: readyTriaged, unblocked, and fully specified; eligible to pick up.

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions