Skip to content

Widen tarfile.open() preset to int - #16443

Merged
srittau merged 1 commit into
python:mainfrom
Sharawey74:fix/tarfile-preset-16442
Sep 25, 2026
Merged

srittau merged 1 commit into
python:mainfrom
Sharawey74:fix/tarfile-preset-16442

Conversation

@Sharawey74

Copy link
Copy Markdown
Contributor

Fixes #16442.

tarfile.open() types preset as Literal[0, 1, 2, 3, 4, 5, 6, 7, 8, 9] | None,
so the documented way to ask for extreme compression is rejected:

tarfile.open("test.tar.xz", mode="w:xz", preset=9 | lzma.PRESET_EXTREME)
# error: No overload variant of "open" matches ...

lzma.PRESET_EXTREME
is a flag you or onto the preset level, so 9 | lzma.PRESET_EXTREME is
9 | 0x80000000 = 2147483657 — not a member of the Literal.
tarfile.open()
passes preset straight through to TarFile.xzopen(), which hands it to
lzma.LZMAFile, so the value it accepts is whatever lzma accepts.

The two sibling APIs already use the lenient type, so this change makes the
three agree:

Location Before After
stdlib/tarfile.pyi — TarFile.open() x2 Literal[0..9] | None int | None
stdlib/tarfile.pyi — TarFile.xzopen() int | None unchanged
stdlib/_lzma.pyi — LZMACompressor int | None unchanged

The trade-off

Widening to int gives up static rejection of out-of-range values. Those now
fail at runtime instead of at type-check time: preset=-1 raises
OverflowError: can't convert negative int to unsigned, and preset=10 raises
LZMAError: Invalid or unsupported options.

That is the same trade-off _lzma.pyi and xzopen() already make, and the
Literal cannot express "0-9, optionally or-ed with PRESET_EXTREME" without
enumerating both ranges. If you would rather keep the narrow form, the
alternative is Literal[0..9] | int spelled as a separate alias — say the word
and I will switch it.

Test case

stdlib/@tests/test_cases/check_tarfile.py asserted the old behaviour:

tarfile.open("test.tar.xz", "w:xz", preset=-1)  # type: ignore
tarfile.open("test.tar.xz", "w:xz", preset=10)  # type: ignore

Those type: ignore comments become unused under the new signature, which
fails regr_test.py. They are replaced with a case covering the actual bug.
Note that mypy --warn-unused-ignores does not surface this when run on files
inside typeshed; it only appears once the test case is copied out, the way
tests/regr_test.py does it.

Verification

Windows 11, mypy 2.3.0, Python 3.12.

tests/mypy_test.py stdlib/tarfile.pyi          success, 6 files checked
tests/mypy_test.py stdlib                      success, 3832 files checked
tests/check_typeshed_structure.py              clean
pyright -p pyrightconfig.json                  0 errors
pyright -p pyrightconfig.stricter.json         0 errors
pyright -p pyrightconfig.testcases.json        0 errors
ty check                                       All checks passed
pyrefly check                                  0 errors
ruff check (stubs)                             All checks passed
ruff check --select=FA,I,ICN001,RUF100         All checks passed
black --check                                  2 files unchanged
flake8 + flake8-pyi                            exit 0
test case copied out, --warn-unused-ignores    Success

Reproduction before the change, and clean after:

import lzma, tarfile
tarfile.open("test.tar.xz", mode="w:xz", preset=9 | lzma.PRESET_EXTREME)

preset="nine" is still rejected.

`tarfile.open()` typed `preset` as `Literal[0, ..., 9] | None`, which
rejects `9 | lzma.PRESET_EXTREME` even though that is the documented way
to request extreme compression. `_lzma.LZMACompressor` and
`TarFile.xzopen()` already type the same parameter as `int | None`, so
the three now agree.

Widening loses the static rejection of out-of-range values: `preset=-1`
raises OverflowError at runtime and `preset=10` raises LZMAError. The
test case is updated to cover the PRESET_EXTREME form instead.

Fixes python#16442
Copilot AI lite review requested due to automatic review settings September 25, 2026 17:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

nionutils (https://github.com/nion-software/nionutils)
+ error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 2.3.0
+ note: use --pdb to drop into pdb
+ Traceback (most recent call last):
+   File "/__main__.py", line 16, in console_entry
+     main()
+     ~~~~^^
+   File "mypy/main.py", line 154, in main
+   File "mypy/main.py", line 244, in run_build
+     res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
+   File "mypy/build.py", line 385, in build
+   File "mypy/build.py", line 1908, in create_metastore
+     mds: MetadataStore = SqliteMetadataStore(
+   File "mypy/metastore.py", line 200, in __init__
+   File "mypy/metastore.py", line 174, in connect_db
+     db.execute("PRAGMA journal_mode=WAL")
+ sqlite3.OperationalError: database is locked
+ 

@srittau
srittau merged commit 249fa03 into python:main Sep 25, 2026
88 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tarfile.open() doesn't account for preset=... | lzma.PRESET_EXTREME

3 participants