Skip to content

Pass 2 summary-capability rule: share one summary sized for the strictest consumer - #595

Draft
zzylol wants to merge 2 commits into
stack/509-w2-executor-panesfrom
stack/509-w7a-pass2-sizing
Draft

zzylol wants to merge 2 commits into
stack/509-w2-executor-panesfrom
stack/509-w7a-pass2-sizing

Conversation

@zzylol

@zzylol zzylol commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Rebased on main d4869a7 (DF 54).

Wave 1 chain: #594 → #593 → #592 → #591 → #595 → #596 → #597 → #598 → #599 (on #589)

Part of #509 (Pass 2), #580 item D, decision W5.

Why

Stage 1 only planned Pass 2's identical-expression rule. Two queries that read the same input with different accuracy requirements, or different quantiles at the same requirement over an unkeyed SQL scan, never shared a summary. #509's summary-capability rule allows one summary sized for the strictest consumer.

What

  • pass2::summary_capability: targets with the same summary input data and window (same input sub-DAG including TimeRange/WHERE, same grouping, same quantile column, approximate requirement) form a key. Every target of a key is re-sized for the strictest requirement (smallest ε and δ), all or nothing per key (W5). Sizing reuses the legacy reconciliation's argument (accuracy_budget; dominates is checked): requirements resolve to (ε, δ) and the sizing formulas are monotonic. First family: quantiles (KLL, DDSketch), so one KLL serves p50 and p99.
  • Stage 1 now has up to three variants, Sharing::{Independent, IdenticalExpressions, SummaryCapability}. SharingVariant.shared: bool is replaced by sharing: Sharing. The capability variant is built on the identical-expression inventory and merges identical producers after composition. It is skipped when it would repeat that variant. Stage 3 checks each query against its own target and chooses.
  • Fix in the tree DP (plan_selection::select_variant): pairs of targets reading a common input were collected but never checked. The loop required one to read the other. Merged producers were therefore missed. The DP now checks those pairs, and compares inputs by value too (SQL scans have no unique key, so CSE does not alias them).
  • Fix in composition: a summary evaluation keeps the measure's explicit SQL output name.

Before / After

quantile(0.5, lat) and quantile(0.99, lat), ε = 0.01, through the facade (stage_pipeline numbers, 15k samples):

selected cost (cpu_ms/eval)
Before P10: two exact quantiles over one shared scan (the DP missed the merge) 0.108
After P14: one KLL, two estimates 0.093

p50 at ε = 0.01 and p99 at ε = 0.001 over lat[5m]: Before, the two KLLs differ (k for 0.01 and for 0.001) and never merge. After, the summary-capability variant sizes both for ε = 0.001, and the plan deploys one KLL (stage_pipeline_shares_one_kll_sized_for_the_strictest_consumer).

Tests

  • Un-ignored in crates/planner/tests/summary_sharing.rs: cross_series_p50_and_p99_share_one_producer, quantiles_with_equal_params_share_one_producer, identical_sql_percentiles_share_one_producer.
  • Still ignored, with new precise reasons:
    • quantiles_share_one_producer_sized_for_the_strictest_consumer: the sharing assertions pass. The pipeline attaches no guarantee to plan roots, and Stage 3 ignores PlanningModels.cost, so alone the looser query selects raw.
    • sql_p50_and_p99_share_one_producer: the shared half now passes, names included. The unshared negative cases select raw, since a query-time summary never costs less until Stage 2 materializes.
  • New: stage_pipeline_shares_one_kll_sized_for_the_strictest_consumer and stage_pipeline_shares_sql_p50_and_p99 cover the passing halves, plus 3 unit tests for the rule and 3 DP-equals-exhaustive tests with sharing (PromQL, PromQL plus an unrelated query, SQL). Two of the DP tests fail without the coupling fix.

Gate

fmt and clippy (-D warnings) are clean. cargo test --workspace: 1,528 passed and 7 ignored, against #589's 1,517 and 10. That is 3 un-ignored plus 8 new tests. The Example 1 fixture regenerates byte-identically through stage_pipeline, because no capability key applies there.

Gaps

  • One variant per workload. Mixed decisions across keys (key A shared, key B independently sized) are not listed.
  • Window-composition rule not planned.
  • Root guarantees are not attached by the pipeline.

🤖 Generated with Claude Code

zzylol and others added 2 commits October 5, 2026 04:48
…riant

Targets with the same summary input data and window (same input, grouping
and quantile column) are re-sized for their strictest consumer, all or
nothing per key (#580 W5), reusing the legacy reconciliation's
accuracy_budget/dominates argument. Stage 1 adds this inventory as a third
variant (Sharing::SummaryCapability) next to the independent and
identical-expression ones; composition then builds identical summary
producers, which are merged, and Stage 3 chooses.

Also:
- the tree DP now checks coupling between targets that read a common (or
  equal) input; it skipped those pairs, so it missed merged producers;
- a composed summary evaluation keeps the measure's explicit output name
  (SQL), instead of the derived one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pass 1 takes the declared metric types since #593, and #594's year-long scan
test selects its variant by the Sharing enum instead of a shared flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/509-w7a-pass2-sizing branch from 3c07c2d to cf0df8a Compare October 5, 2026 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant