Skip to content

[agentic-token-optimizer] Optimize PR Sous Chef: reduce AIC via duplicate-line cleanup, redundant skip-table removal, and working-set growth mitigation #63459

Description

@github-actions

Target workflow

PR Sous Chef (.github/workflows/pr-sous-chef.md) — highest total AIC among workflows not optimized in the last 14 days (324.4 total AIC over 11 runs in the last 7 days; avg 29.5/run). Runner-up candidates (Claude Code User Docs Review, Repository Quality Improvement Agent, Daily File Diet) were single-run outliers or already optimized recently.

Analysis period

Last 7 days, 11 runs analyzed (10 success, 1 failure — agent_logic failure kind, run §36149578895).

Cost profile

Metric Value
Total AIC (11 runs) 324.41
Avg AIC/run 29.49
Total raw tokens 5,404,980
Avg raw tokens/run ~491,362
Avg turns/run 0 (not populated)
Avg sub-agent invocations/run 11.9
Avg working-set rebuild factor 9.0x
Avg cumulative input tokens/run 488,827
Total GitHub API calls ~220 (≈20/run)
Safe items produced (11 runs) 17

Ranked recommendations

1. Remove duplicated instruction line (~2–3% AIC savings/run, ~0.6–0.9 AIC/run)

Evidence: Line 466–467 of the prompt body contains an exact duplicate sentence:

- Every `add_comment` must include `pr_number`. Never emit `add_comment` without a numeric target field.
- Every `add_comment` must include `pr_number`. Never emit `add_comment` without a numeric target field.

This line is re-sent as static prompt context on every one of the 11 runs analyzed with no functional benefit — pure duplicated tokens. Action: delete one of the two identical lines.

2. Finish the remove_redundant_context_v1 experiment rollout (~3–5% AIC savings/run once graduated)

Evidence: The workflow already has a live experiment (remove_redundant_context_v1) testing removal of the "Required skip rules per PR" backup table, on the stated rationale that the same 3 conditions are already deterministically enforced by the fetch-prs prefilter step and re-verified by the pr-processor sub-agent — i.e., the table is redundant on every run regardless of variant. The experiment requires min_samples: 20; only 11 runs have been observed in the analysis window. Action: no prompt change recommended until the experiment reaches its min_samples threshold and a decision is made per its existing mann_whitney analysis — flagging this here only to confirm the audit is aware of the in-flight test and does not duplicate or contradict it.

3. Investigate working-set rebuild-factor variance (~5–10% AIC savings/run on high-rebuild runs)

Evidence: Rebuild factor ranges from 4.3x to 14.6x across the 11 runs, with the two highest-AIC runs (48.9 AIC / 21 invocations / 14.6x rebuild; 32.6 AIC / 16 invocations / 13.4x rebuild — the latter being the failed run) also showing the highest sub-agent invocation counts and cumulative input tokens (867K–927K). Runs with fewer eligible/nudged PRs (safe_items_count=1) cluster around 4–7x rebuild factor and 6–9 invocations. This suggests each additional PR processed via the pr-processor sub-agent re-sends a large amount of shared context rather than reusing it. Action: confirm the pr-processor sub-agent invocation only passes "the PR number and compact context" as instructed (step 7 already mandates this) — audit whether the compact JSON payload size grows with PR count, and consider trimming failed_checks/reviewThreads fields passed per-PR to only what's needed for the current PR being processed.

4. Investigate the single agent_logic failure (~9% of runs at 100% partial-completion risk)

Evidence: Run §36149578895 completed 5 safe outputs (2× update_pull_request, 2× add_comment, 1× create_issue) but was still marked conclusion: failure with failure_kind: agent_logic and error_count: 1, at the highest token cost of the sample (678,188 tokens, 32.6 AIC). Wasted spend on a failed run compounds cost without full nudge value delivered. Action: pull the run's error/warning log for this specific run to identify the root cause (likely a missing precondition check or unhandled sub-agent response) and add a targeted guard.

Caveats

  • Sample size is limited to 11 runs in a 7-day window; turns field is not populated (0 for all runs), so turn-based iteration analysis was not possible — recommendations rely on AIC, token, and working-set metrics instead.
  • The workflow already has one active experiment in flight (remove_redundant_context_v1); this audit intentionally avoids proposing an overlapping change to the same section.
  • No new inline sub-agents are recommended: the workflow already has one (pr-processor) handling the clear per-PR-independent task, and no other section met the "no existing agent: blocks" precondition for additional inline sub-agent recommendations.
  • The single failed run's root cause was not fully diagnosed here (would require deeper log inspection); flagged as recommendation Add workflow: githubnext/agentics/weekly-research #4 for follow-up rather than a confirmed fix.
Raw per-run metrics (11 runs)
AIC Invocations Rebuild factor Cumulative input tokens API calls Safe items Conclusion
32.61 16 13.35x 675,668 20 5 failure
36.72 15 12.78x 545,288 20 1 success
18.94 10 7.53x 315,659 22 1 success
30.88 13 10.75x 521,920 18 1 success
21.15 6 4.29x 241,474 20 1 success
48.89 16 11.80x 926,833 20 2 success
23.91 7 4.48x 261,839 20 1 success
10.17 6 4.68x 115,124 22 1 success
43.96 21 14.59x 867,679 18 1 success
31.59 12 9.15x 548,628 20 1 success
25.58 9 5.64x 356,987 20 2 success

References: §36149578895 (failed run), §36147251830, §36140522853 (highest AIC success run)

Generated by Agentic Workflow AIC Usage Optimizer · copilot · auto · 72.7 AIC · ⊞ 10.8K · ◷

  • expires on Oct 2, 2026, 7:02 AM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions