perf: remove duplicate case-row materialization

[TonisOrmisson]

Summary

Phase 7a: remove one redundant Python case-row materialization from the existing read/validate/export path.

• Feed the ordered SQLAlchemy mapping result directly into the existing numeric normalizer.
• Keep the normalizer's final list[dict] result and every public signature; this removes only the preliminary list of copied dictionaries.
• Preserve snapshot lifetime, read-only behavior, ordering, binary64/NULL/string values, metadata, diagnostics, publication and writer behavior.
• No full streaming API, cursor mode, import/batch rewrite, PHP change, dependency, configuration or persistent copy.

RED/GREEN and checks

• RED 4981356: three populated public surfaces failed solely because 4 cases created 8 dictionaries; empty variants already created 0.
• GREEN efeda69: production only; tests immutable after RED.
• Focused coordinator run: 125 passed, 9 skipped, 8 deselected.
• Full non-service implementation run: 408 passed, 9 skipped, 53 deselected; evidence rerun excluding the existing release-reference test: 403 passed, 9 skipped, 53 deselected. The excluded test creates temporary Git repositories/configuration as part of its own existing behavior.
• Independent read-only full-diff review: No findings.
• All 29 GitHub checks successful, merge state CLEAN. CodeRabbit completed after rate-limit reset with no actionable comments/minimal merge risk; no review threads. Its generic docstring-percentage warning is non-actionable repository boilerplate, not a missing contract.
• @codex review was requested at 5598078269 and retried at 5598267518, but the service returned an account usage-limit message (5598080787) rather than running a review. No Codex result is claimed.

Evidence

Immutable base 06b346ae86247fa88d65529398b9c5b0c25d5a10 versus head efeda69309dbdb317e8d74334c5b501292eb9635, same Python 3.13.2 / SQLAlchemy 2.0.52 / SQLite 3.47.1 / pandas 3.0.5 / pyspssio 0.5.1.post2.

Public surface	Case dictionaries base → after	Empty base → after
read_wide_dataset	8 → 4	0 → 0
validate	8 → 4	0 → 0
real SAV export/decode	8 → 4	0 → 0

Rows, metadata, validation results, SQL ordering, read-only database checksum and decoded SAV output matched exactly; binary64 bits, NULL, empty/UTF-8 strings, labels, missing rules, formats and attributes were checked. No SAV-byte identity claim.

Fixed recursive read fragment allocation measurements (not full-operation claims):

Rows	Simultaneously live dictionaries	Tracemalloc peak	Reduction
10,000	20,000 → 10,000	12.97 → 8.24 MB	4.72 MB / 36.4%
100,000	200,000 → 100,000	129.49 → 82.29 MB	47.20 MB / 36.4%

The final public result remains buffered, as do pandas and the writer. No full-streaming or server-driver RSS claim is made. Raw commands, source manifests and limits: ../memory-bounds-evidence.md and /tmp/phase7a-evidence-20260909, outside the repository.

Scope and next

Python import/DataFrame copies, PHP exporter memory, SQLAlchemy unbuffered cursors and full streaming are deferred; they require separate evidence/API decisions. After user merge: continue the remaining approved Phase 7 measurement only if another safe, proven slice exists. Overall phases 0–9 remain unfinished. Do not auto-merge.

[TonisOrmisson]

@codex review

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: cd1e8944-6219-4968-aa8f-e1d1434188ac

📥 Commits

Reviewing files that changed from the base of the PR and between 06b346a and efeda69.

📒 Files selected for processing (2)

• src/openstatspec/sql/wide.py
• tests/test_read_only_export.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The wide dataset reader now passes database mappings directly to numeric row canonicalization. SQLite tests cover mixed values, non-ASCII names, empty datasets, and single-copy behavior across descriptor, validation, and export operations.

Changes

Wide dataset read optimization

Layer / File(s)	Summary
Direct row normalization src/openstatspec/sql/wide.py	_read_wide_dataset passes the .mappings() iterable directly to _canonicalize_database_numeric_rows without creating an intermediate list.
Read operation coverage tests/test_read_only_export.py	SQLite fixtures add mixed and empty datasets. Parametrized tests verify returned rows, diagnostics, metadata, and one case-dictionary copy per read operation.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to efeda

Wide dataset reads now avoid a redundant intermediate row allocation while preserving normalized read, validation, and export results. No merge-blocking production risk is identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main performance change: removing duplicate case-row materialization.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/memory-bounds

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[TonisOrmisson]

@coderabbitai review

[TonisOrmisson]

@codex review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[TonisOrmisson]

@codex review

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.