Conversation
This was referenced Oct 4, 2026
A SummaryMerge now runs when its consumer runs, like other state operators, and reads summary state from its inputs; a merge at ingestion time rejects query-time inputs. apply_materialization_timings previously rejected it as UnimplementedOperator, so planner-emitted tumbling panes (#580) could not be exported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The #509 window-composition rule's tumbling windows, as an axis on Pass 1 alternatives (#580 W1-W4). For a repeating query over a range window, each alternative whose family merges also comes in panes of width gcd(lookback, cadence), the cadence taken from the root's RootDemand: N pane SummaryAggs, pane i over TimeRange(w) over TimeShift(o + i*w) over the scan with coverage RelativeToEvaluation(-(o+(i+1)w)..-(o+iw)), merged by a SummaryMerge before the estimate or finalize. Stage 1 offers the form only when the panes' merged coverage is exactly the whole window. - Stage 3 checks accuracy through a merge on each pane build (merged KLL/DDSketch keep the family's analytical guarantee) and prices a merge per input state, its output as the union of the inputs' groups. - The DP also checks targets whose panes read one scan for coupling. - FinalizeExactAccumulator over a SummaryMerge keeps the finalized value's column name (it was "state", so readers could not resolve it). - Example 1 gains Q2's exact sum in 10-s panes: 64 -> 88 candidates, same selected plan (now P82) and cost; the fixture is generated with --max-candidates 128. The devtool labels window forms ("Kll · tumbling 1m panes") and adds --example planner-layering-3a and -3b (#590). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ports the #590 Example 3 acceptance tests and their common module onto the stage pipeline with root demand. window_form reads a pane (a build merged by a SummaryMerge) as Tumbling of its TimeRange width. Adds Pattern B tests for the tumbling forms, their five panes and coverage, Stage 3 validity, and a runtime check that the planner's tumbling KLL plan executes and matches the whole-window plan. The shared-scan cost test now passes and is un-ignored; EH, partial groupings and sliding windows stay ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/509-x1-stage1-tumbling
branch
from
October 5, 2026 06:21
4adff80 to
a118f42
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased on main d4869a7 (DF 54).
Stacked on #599 (wave 1 chain). Part of #580 item A, step 4; #509 Pass 2 window-composition rule.
Why
#509 Example 3, Pattern B, answers a 5-min p99 that runs every minute by merging 1-min tumbling panes. #591 made the executor run that shape, but the planner never generated it.
apply_materialization_timingsalso rejectedSummaryMerge, so a pane plan could not be exported.What
SummaryMergeruns at its consumer's timing. Its inputs must be summary state, and a merge at ingestion time rejects query-time inputs.WindowForm::{Whole, Tumbling { pane_ms }}is set per alternative (LocalLogicalTarget::windows). A query over a range window that repeats on a cadence gets a tumbling copy of every alternative whose family merges (family_merges()), withpane_ms = gcd(lookback, cadence). The cadence comes fromRootDemand. A form needs 2 to 64 panes.stage1_logical_candidatesnow takesdemand. Labels read "Q1 Kll · tumbling 1m panes".SummaryAggoverTimeRange(w)overTimeShift(o + i·w)over the oneScan, where o is the query's own offset. It coversRelativeToEvaluation(-(o+(i+1)w) .. -(o+iw)). ASummaryMergecombines the panes beforeSummaryEstimateorFinalizeExactAccumulator. Stage 1 offers the form only if the panes' merged coverage equals the whole window[-(o+lookback), -o). The site detection is ported from Generate exact window composition and materialization candidates #566compose().accuracy_violationlooks through a merge and checks each pane build against the family's analytical guarantee. A merge is priced at one operation per input state. Its output is the union of the inputs' groups, bounded the way one build is. This matters: without it, a reader of the merge was priced on one pane's groups, and the DP saw a spurious coupling.FinalizeExactAccumulatorover aSummaryMergenamed its value columnstate, so readers such as a heap sketch oversum_over_timefailed to compile. It now looks through the merge.Before / After
sum_over_timealso comes in 10-s panes; rates and heaps do not mergeTumbling costs more for now because everything runs at query time, so every pane is rebuilt at each evaluation. In 3B, the five pane builds together cost the same as the whole build (5 × 0.067 = 0.333). The extra +3.08 comes from each pane's
TimeShiftandTimeRangepassing the full 5-min scan, 20M rows (5 × 0.667 against 0.333), plus 0.083 for the merge. This is left untuned, as asked.The Example 1 fixture is now 9.3 MB (was 4.1 MB). It has 88 candidates, above the devtool's default display cap of 64, so it is generated with
--max-candidates 128. The selection fallback cap stays 64, and the facade's DP (#572) still returnsTreeDpon Example 1.Tests
SummaryMergetiming (regression for fix 1).planner_layering_common(window_formreads a merged build as a pane). The tumbling-only tests pass.stage3_a_shared_scan_is_not_costliernow passes and is un-ignored. Five tests stay ignored: EH ×2, partial groupings, and sliding windows ×2; the sliding ones name the tumbling test that passes. The spec doc stays in test: #509 Examples 2–4 acceptance specs and end-to-end tests #590.stage_pipeline --example planner-layering-3a|3badded (from test: #509 Examples 2–4 acceptance specs and end-to-end tests #590). The Example 1 fixture is regenerated.Gate:
cargo fmt --all --check,cargo clippy --workspace --all-targets --all-features -- -D warnings,cargo test --workspace: 1,609 passed / 12 ignored (#599: 1,580 / 7). Viewer tests: 29 OK.What Stage 2 materialization needs from this PR
Panes are separate
SummaryAggnodes, each with evaluation-relative coverage, so Stage 2 can assign each one ingestion time (B1) or query time kept (B3) throughMaterializationAssignment::set. The merge then follows its consumer. Retention islookbackper pane width. Nothing marks yet which pane is the newest one to build at query time (B3); its coverage[-w, 0)identifies it.Links: #509, #580, #566 (
compose()origin), #591 (executor pane merges), #590 (Example 3 acceptance tests).🤖 Generated with Claude Code