You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PR Sous Chef (.github/workflows/pr-sous-chef.md) — highest total AIC among workflows not optimized in the last 14 days (324.4 total AIC over 11 runs in the last 7 days; avg 29.5/run). Runner-up candidates (Claude Code User Docs Review, Repository Quality Improvement Agent, Daily File Diet) were single-run outliers or already optimized recently.
Analysis period
Last 7 days, 11 runs analyzed (10 success, 1 failure — agent_logic failure kind, run §36149578895).
Cost profile
Metric
Value
Total AIC (11 runs)
324.41
Avg AIC/run
29.49
Total raw tokens
5,404,980
Avg raw tokens/run
~491,362
Avg turns/run
0 (not populated)
Avg sub-agent invocations/run
11.9
Avg working-set rebuild factor
9.0x
Avg cumulative input tokens/run
488,827
Total GitHub API calls
~220 (≈20/run)
Safe items produced (11 runs)
17
Ranked recommendations
1. Remove duplicated instruction line (~2–3% AIC savings/run, ~0.6–0.9 AIC/run)
Evidence: Line 466–467 of the prompt body contains an exact duplicate sentence:
- Every `add_comment` must include `pr_number`. Never emit `add_comment` without a numeric target field.
- Every `add_comment` must include `pr_number`. Never emit `add_comment` without a numeric target field.
This line is re-sent as static prompt context on every one of the 11 runs analyzed with no functional benefit — pure duplicated tokens. Action: delete one of the two identical lines.
2. Finish the remove_redundant_context_v1 experiment rollout (~3–5% AIC savings/run once graduated)
Evidence: The workflow already has a live experiment (remove_redundant_context_v1) testing removal of the "Required skip rules per PR" backup table, on the stated rationale that the same 3 conditions are already deterministically enforced by the fetch-prs prefilter step and re-verified by the pr-processor sub-agent — i.e., the table is redundant on every run regardless of variant. The experiment requires min_samples: 20; only 11 runs have been observed in the analysis window. Action: no prompt change recommended until the experiment reaches its min_samples threshold and a decision is made per its existing mann_whitney analysis — flagging this here only to confirm the audit is aware of the in-flight test and does not duplicate or contradict it.
Evidence: Rebuild factor ranges from 4.3x to 14.6x across the 11 runs, with the two highest-AIC runs (48.9 AIC / 21 invocations / 14.6x rebuild; 32.6 AIC / 16 invocations / 13.4x rebuild — the latter being the failed run) also showing the highest sub-agent invocation counts and cumulative input tokens (867K–927K). Runs with fewer eligible/nudged PRs (safe_items_count=1) cluster around 4–7x rebuild factor and 6–9 invocations. This suggests each additional PR processed via the pr-processor sub-agent re-sends a large amount of shared context rather than reusing it. Action: confirm the pr-processor sub-agent invocation only passes "the PR number and compact context" as instructed (step 7 already mandates this) — audit whether the compact JSON payload size grows with PR count, and consider trimming failed_checks/reviewThreads fields passed per-PR to only what's needed for the current PR being processed.
4. Investigate the single agent_logic failure (~9% of runs at 100% partial-completion risk)
Evidence: Run §36149578895 completed 5 safe outputs (2× update_pull_request, 2× add_comment, 1× create_issue) but was still marked conclusion: failure with failure_kind: agent_logic and error_count: 1, at the highest token cost of the sample (678,188 tokens, 32.6 AIC). Wasted spend on a failed run compounds cost without full nudge value delivered. Action: pull the run's error/warning log for this specific run to identify the root cause (likely a missing precondition check or unhandled sub-agent response) and add a targeted guard.
Caveats
Sample size is limited to 11 runs in a 7-day window; turns field is not populated (0 for all runs), so turn-based iteration analysis was not possible — recommendations rely on AIC, token, and working-set metrics instead.
The workflow already has one active experiment in flight (remove_redundant_context_v1); this audit intentionally avoids proposing an overlapping change to the same section.
No new inline sub-agents are recommended: the workflow already has one (pr-processor) handling the clear per-PR-independent task, and no other section met the "no existing agent: blocks" precondition for additional inline sub-agent recommendations.
The single failed run's root cause was not fully diagnosed here (would require deeper log inspection); flagged as recommendation Add workflow: githubnext/agentics/weekly-research #4 for follow-up rather than a confirmed fix.
Target workflow
PR Sous Chef (
.github/workflows/pr-sous-chef.md) — highest total AIC among workflows not optimized in the last 14 days (324.4 total AIC over 11 runs in the last 7 days; avg 29.5/run). Runner-up candidates (Claude Code User Docs Review, Repository Quality Improvement Agent, Daily File Diet) were single-run outliers or already optimized recently.Analysis period
Last 7 days, 11 runs analyzed (10 success, 1 failure —
agent_logicfailure kind, run §36149578895).Cost profile
Ranked recommendations
1. Remove duplicated instruction line (~2–3% AIC savings/run, ~0.6–0.9 AIC/run)
Evidence: Line 466–467 of the prompt body contains an exact duplicate sentence:
This line is re-sent as static prompt context on every one of the 11 runs analyzed with no functional benefit — pure duplicated tokens. Action: delete one of the two identical lines.
2. Finish the
remove_redundant_context_v1experiment rollout (~3–5% AIC savings/run once graduated)Evidence: The workflow already has a live experiment (
remove_redundant_context_v1) testing removal of the "Required skip rules per PR" backup table, on the stated rationale that the same 3 conditions are already deterministically enforced by thefetch-prsprefilter step and re-verified by thepr-processorsub-agent — i.e., the table is redundant on every run regardless of variant. The experiment requiresmin_samples: 20; only 11 runs have been observed in the analysis window. Action: no prompt change recommended until the experiment reaches itsmin_samplesthreshold and a decision is made per its existingmann_whitneyanalysis — flagging this here only to confirm the audit is aware of the in-flight test and does not duplicate or contradict it.3. Investigate working-set rebuild-factor variance (~5–10% AIC savings/run on high-rebuild runs)
Evidence: Rebuild factor ranges from 4.3x to 14.6x across the 11 runs, with the two highest-AIC runs (48.9 AIC / 21 invocations / 14.6x rebuild; 32.6 AIC / 16 invocations / 13.4x rebuild — the latter being the failed run) also showing the highest sub-agent invocation counts and cumulative input tokens (867K–927K). Runs with fewer eligible/nudged PRs (safe_items_count=1) cluster around 4–7x rebuild factor and 6–9 invocations. This suggests each additional PR processed via the
pr-processorsub-agent re-sends a large amount of shared context rather than reusing it. Action: confirm thepr-processorsub-agent invocation only passes "the PR number and compact context" as instructed (step 7 already mandates this) — audit whether the compact JSON payload size grows with PR count, and consider trimmingfailed_checks/reviewThreadsfields passed per-PR to only what's needed for the current PR being processed.4. Investigate the single agent_logic failure (~9% of runs at 100% partial-completion risk)
Evidence: Run §36149578895 completed 5 safe outputs (2×
update_pull_request, 2×add_comment, 1×create_issue) but was still markedconclusion: failurewithfailure_kind: agent_logicanderror_count: 1, at the highest token cost of the sample (678,188 tokens, 32.6 AIC). Wasted spend on a failed run compounds cost without full nudge value delivered. Action: pull the run's error/warning log for this specific run to identify the root cause (likely a missing precondition check or unhandled sub-agent response) and add a targeted guard.Caveats
turnsfield is not populated (0for all runs), so turn-based iteration analysis was not possible — recommendations rely on AIC, token, and working-set metrics instead.remove_redundant_context_v1); this audit intentionally avoids proposing an overlapping change to the same section.pr-processor) handling the clear per-PR-independent task, and no other section met the "no existing agent: blocks" precondition for additional inline sub-agent recommendations.Raw per-run metrics (11 runs)
References: §36149578895 (failed run), §36147251830, §36140522853 (highest AIC success run)