Repository navigation
Conversation
4 tasks done
zzylol
force-pushed
the
stack/windows-1-b3
branch
from
October 5, 2026 03:07
5540b73 to
9a2a154
Compare
zzylol
force-pushed
the
stack/windows-2-segments
branch
from
October 5, 2026 03:07
7148bf3 to
14921d0
Compare
This was referenced Oct 5, 2026
Pass 2's window-composition rule gains shared segments: approximate single-estimate targets over windows of one scan (same estimate up to accuracy, same grouping, no filters) share one summary per segment of a grid whose width is the gcd of every window's lookback and offset. Each query merges the segments its range covers; composition builds identical segments, which the identical-expression rule merges. Merging disjoint KLLs is exact, so each segment is sized for the strictest consumer and no accuracy is split. Only the all-shared form is offered (Q62), as its own Stage 1 variant (Sharing::WindowSegments, "· shared segments") in which each grouped target has the segment form as its only alternative. A group is skipped when its windows are identical, the union has more than MAX_PANES segments, or its queries recur at different cadences. Example 3a gets one candidate with five 1-year KLL segments; 4a, repeated monthly, can also maintain them at ingestion time (Q61). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/windows-1-b3
branch
from
October 5, 2026 06:21
9a2a154 to
aa2ddc0
Compare
zzylol
force-pushed
the
stack/windows-2-segments
branch
from
October 5, 2026 06:21
14921d0 to
29b20b8
Compare
This was referenced Oct 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: #574 → #620 → #618 → #621 → #627 → #625 → #628 → #632 → #634 → #616 → #617 → #622 → #624 → #629 → #630 → #631 → #633 → #635 → #636 → #637
Problem
Example 3's Pattern A (five p99 reports over [5y], [1y], [1y offset 1y], [1y offset 2y] and [3y offset 2y]) and Example 4a (the same, repeated monthly) had no plan that shares window summaries across the reports. The spec's Exponential Histogram tests were ignored.
Changes
Stacked on #625. Decisions Q60–Q62.
share_window_segmentsinwindow_composition.rsgroups approximate single-estimate targets over windows of one scan: same estimate up to accuracy, same grouping, no filters, offset ≥ 0. The segment width is the gcd of every window's lookback and offset, so every window boundary lies on the grid. For Pattern A that gives five 1-year segments.WindowForm::Segments). Coverage is proved bySummaryCoverage::merge_disjoint, as for panes.strictestandwith_accuracy.Sharing::WindowSegments, labeled· shared segments. In it each grouped target has the segment form as its only alternative, so Pattern A gains exactly one logical candidate (486 → 487). There are no partial or pairwise groupings.MAX_PANES(64) segments, or when its queries recur at different cadences.duration_labelgains days, so segments read365d segments.Tests
stage1_a_window_composition_adds_one_eh_for_all_fivebecomesstage1_a_shared_segments_serve_all_five: five 1-year KLL segments together read by all five queries, each query covering as many segments as its range has years, and five p99 estimates each reading a merge.stage1_a_keeps_independent_and_shared_window_summaries(Example 3).stage2_a_materialized_eh_is_built_once_for_all_consumersbecomesstage2_a_segments_are_built_once_for_all_consumers: five builds read by all five queries through five merges, charged once. Its options are {not materialized} for the one-off batch and {ingestion time, not materialized} for monthly.stage1_a_window_composition_groups_every_pair: no partial or pairwise groupings (Q62).stage2_a_shared_eh_has_three_materialization_options,stage2_a_at_rest_drops_the_ingestion_time_option,stage3_a_rebuilding_per_query_costs_more_than_building_once,stage3_a_once_adhoc_prefers_the_query_time_ehandstage3_a_monthly_amortizes_ingestion_time_maintenance. These expect A1 (kept for one batch), A3 (Q44) or a one-off ingestion-time option (Q61), none of which is generated.pattern_a_shares_five_one_year_segmentsandsegments_need_one_cadence_and_different_windows, unit tests in Pass 2.shares_segmentsin the test adapter.example4a_repeats_monthly_with_nothing_maintainablebecomes..._with_only_segments_maintainable: 488 plans, the only ingestion-time one being the segments.Finding: segments do not win under today's cost model
The 3a segment plan costs 44 384/s against 39 128/s for the selected shared-input KLLs. In 4a it is 61.64/s at query time and 1 536.9/s at ingestion time, against 54.34/s.
The plan builds 5 years of rows into KLLs where the independent KLLs build 11 years. It loses on pass-through work instead: Stage 3 charges each query-time
TimeShiftandTimeRangeper row of its input, and every segment's shift sees the whole 5-year scan. That is 10 such nodes against 8 in the selected plan, at 2 920 each.Pricing a query-time shift as free, and a range on the rows it keeps (a time-ordered scan seeks), would make segments cheapest in 3a: roughly 18 100 against 25 100. That pricing change is not made here.
Test plan
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspace: 1689 passed, 0 failed, 13 ignored (re-run after the rebase on main d4869a7)python3 -m unittest test_viewer(tools/dag-viewer): 25 OK🤖 Generated with Claude Code