Repository navigation

Zarrquet implementation: Adding Zarrs-based TableProvider with bindings in Python. - #69

Closed

alxmrs wants to merge 65 commits into

mainfrom

Commits on Jul 15, 2025
- Integrated rust into the project via maturin
  alxmrs
  committed
  6f9fd56
Commits on Jul 16, 2025
Commits on Jul 17, 2025
- Generated integration tests to test sql functionality.
  alxmrs
  committed
  b6efef2
- Initial `from_zarr` implementation, which calls the rust TableProvider, with unit tests.
  alxmrs
  committed
  9e6a0b0
- First stage of multi-stage test refactoring from Claude: creating a test base.
  alxmrs
  committed
  1937393
- Made some minor refactors.
  alxmrs
  committed
  6457dd8
- Implemented strategy 2 and 3.
  alxmrs
  committed
  0777064
- Import opt.
  alxmrs
  committed
  7a5700a
- Tests for semantic equivalence to xarray operations.
  alxmrs
  committed
  624c3af
- Moved test data creation scripts into examples.
  alxmrs
  committed
  7c328e0
- Removing examples for now, because they add too much noise to the review.
  alxmrs
  committed
  028e69b
- Reformatted the python code with pyink.
  alxmrs
  committed
  b30bde6
- Import optimization
  alxmrs
  committed
  fff4203
- Revert to simpler tests! These tests have gotten too complicated and illegible.
  alxmrs
  committed
  0dd67fd
- WIP, manually refactoring tests to test both from_zarr and from_dataset
  alxmrs
  committed
  cd59d51
- I think I found an actual error with the Zarr logic.
  alxmrs
  committed
  27baf67
- Fixed signficant correctness bugs in the tabel provider impl related to corrent column names as well as correct scanning (type handing).
  alxmrs
  committed
  38f2796
- Test is incorrect; updating.
  alxmrs
  committed
  fdf19bf
- Table provider passes tests, but it looks like it needs to be simplified!
  alxmrs
  committed
  8476f5f
- Continuing my zarr vs dataset refactor.
  alxmrs
  committed
  76196d6
- Pinning down the first of a two part problem.
  alxmrs
  committed
  c2d95c2
- Added skips to tests to look into later.
  alxmrs
  committed
  acbb2bb
- Reformatted tests.
  alxmrs
  committed
  6e02906
Commits on Jul 18, 2025
- Attempt to fix minor warnings.
  alxmrs
  committed
  c85d382
Commits on Jul 22, 2025
Commits on Jul 24, 2025
- Apply clippy suggestions and rustfmt to #69 (#70)
  kylebarron
  authored
  8b6beb5
Commits on Jul 25, 2025
Commits on Jul 27, 2025
- `from_zarr` is now only supported for Python 3.11+
  alxmrs
  committed
  7a3ca51
- pyink fmt
  alxmrs
  committed
  43e5046
- Try to fix linker problem for the maturin arm build.
  alxmrs
  committed
  92203fe
- Put in the wrong file
  alxmrs
  committed
  7e20540
- Upgrade ubuntu version to get gcc 14
  alxmrs
  committed
  29d132e
- Attempting to fix IBM target via: https://github.com/rust-lang/stacker/issues/79#issuecomment-1497479267
  alxmrs
  committed
  986b498
Commits on Jul 29, 2025
- Add test to create test datasets for rust tests.
  alxmrs
  committed
  ea6870a
- Adding integration tests from Claude.
  alxmrs
  committed
  b7a8494
- Adding build step to run integration tests
  alxmrs
  committed
  69f065a
- fmt int tests
  alxmrs
  committed
  243015b
Commits on Sep 11, 2025
- Attempting to take a much simpler approach via an adapter.
  alxmrs
  committed
  14b4ea2
Commits on Sep 12, 2025
- New, simpler approach that makes use of `arrow-zarr`.
  alxmrs
  committed
  5ac86e4
Commits on Sep 26, 2025
- Current WIP, got back to where I started :/
  alxmrs
  committed
  82e54cc
- Fix: don't pass chunks.
  alxmrs
  committed
  7f24256
Commits on Sep 27, 2025
- Made a really important fix! The python datafusion version needs to match rust!!
  alxmrs
  committed
  1d58461
- Updated xarray-sql branch of arrow-zarr.
  alxmrs
  committed
  76102d9