Skip to content

docs: define unified operators and SQL/PromQL scalar boundaries - #511

Merged
zzylol merged 50 commits into
mainfrom
docs/operator-flattening-proposal
Oct 2, 2026
Merged

zzylol merged 50 commits into
mainfrom
docs/operator-flattening-proposal

Conversation

@Selvomega

@Selvomega Selvomega commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Define a common operator graph before and after ASAP optimization, and separate query operators from scalar expressions using explicit SQL/PromQL semantic mappings.

Before this PR

Relational subplans are hidden behind wrappers, and ordinary operations have separate representations above and below summaries. The scalar/operator proposal largely copies the current mixed representation without establishing language coverage.

After this PR

  • Define Operator::NonASAP and Operator::ASAP, with a common OperatorNode carrying result kind, schema, guarantee and timing. Keep one canonical set of operation payloads across both documents; distinguish structure and timing validation from planner accuracy assessment.
  • Define NonASAPOp and ScalarExpr against DataFusion 55.1.0 and Prometheus 3.15.0, with tables for direct mappings, equivalent lowerings and remaining semantic gaps.
  • Remove PromqlScalarBridge: for example, up * 2 uses a projection with a scalar literal. Retain real scalar/vector conversion and SQL subquery semantics, with visible operator dependencies.
  • Specify comparison modes, temporal selection, standalone scalar queries and evaluation context without claiming complete language or runtime support.
  • Show a complete SQL SUM graph before and after an exact-accumulator rewrite, including value/state schema changes.
  • Preserve existing accuracy rules and follow docs: propose workload-wide planning, summary sharing, and materialization #509 for timing and planning responsibilities. Export the common graph without relational wrappers.

Documentation only. The repository still depends on DataFusion 43.0.0 and a PromQL parser declaring 3.8 compatibility; this PR does not upgrade runtime dependencies.

Validation

Reviewed the pinned upstream definitions and language references. Checked both documents' operator lists, scalar placement, links, references, code fences and whitespace. Compiled the combined Rust structure sketches with placeholders for existing payloads; checked that type definitions are unique and operation lists remain complete. No runtime tests run.

Selvomega and others added 8 commits September 28, 2026 02:07
Add a design proposal for merging the pre- and post-ASAP operator sets
into one Operator { Basic(OriginalOp), Ext(ASAPOp) } tree, and list it
in the proposals index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both proposal documents take main's version: the branch content was
squash-merged as #469 and main then updated the file and function names
in #470.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Comment thread docs/design_docs/proposals/operator-sharing.md Outdated
@Selvomega Selvomega changed the title Docs/operator flattening proposal updated docs: operator flattening proposal updated Oct 1, 2026
@zzylol zzylol changed the title docs: operator flattening proposal updated docs: clarify operator sharing, lifecycle timing, and DAG export Oct 1, 2026
@zzylol zzylol changed the title docs: clarify operator sharing, lifecycle timing, and DAG export docs: explain the unified operator design and its rationale Oct 1, 2026
@zzylol
zzylol deleted the docs/operator-flattening-proposal branch October 2, 2026 17:22
This was referenced Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants