Skip to content

Stream query execution from the library - #32

Closed
donatj wants to merge 1 commit into
masterfrom
fix/stream-query-execution
Closed

donatj wants to merge 1 commit into
masterfrom
fix/stream-query-execution

Conversation

@donatj

@donatj donatj commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Query execution lived in the command package. It coordinated two producers through buffered channels. It could exit the process on a parse error. It did not own worker completion consistently.

This extracts ExecuteQuery into the library. It streams selected rows through a callback. The command remains responsible for CSV output and flushing. The executor owns the active lexer worker. It stops that worker and waits before returning.

The executor accepts the query limit data model. User-query LIMIT syntax remains in the stacked PR.

The data-section lexer now uses a 64-token queue. Read, parse, callback, and CSV flush errors return to the caller. The row parser has a synchronous callback path. This keeps the Pike-style state functions while reducing retained work for small results.

Tests cover scanning, indexing, execution across INSERT blocks, offsets, zero limits, and output errors. The benchmark constructs queries directly so it measures execution without user-query parsing.

Validation:

  • go test ./... -timeout 30s
  • go test -race ./... -timeout 60s
  • go build ./...
  • go vet ./...
  • go test ./cmd/sqlread -run '^$' -bench '^BenchmarkExecQuery$' -benchmem -benchtime=200ms -count=1

@donatj

donatj commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

Closed while restoring #30 to its original unstacked form.

@donatj donatj closed this Oct 11, 2026
@donatj
donatj deleted the fix/stream-query-execution branch October 11, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant