Skip to content

gh-133998: Fix tarfile error on out-of-range mtime in streaming gzip mode - #151828

Open
harjothkhara wants to merge 3 commits into
python:mainfrom
harjothkhara:gh-133998-tarfile-mtime
Open

harjothkhara wants to merge 3 commits into
python:mainfrom
harjothkhara:gh-133998-tarfile-mtime

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

A streaming gzip tarball ("w|gz") with an out-of-range mtime (negative, or a clock past 2106) raises struct.error instead of writing the file.

gzip already clamps these to 0 (GH-134278); this does the same for tarfile's streaming path. The non-streaming w:gz path uses GzipFile, so it was already fine.

Added tests for the boundary and bad-clock cases.

AI-assisted; I reviewed it and can explain it.

…rites

tarfile.open(..., "w|gz", mtime=...) packed mtime into the gzip header's
32-bit field with no range check, so a value < 0 or >= 2**32 raised
struct.error. Mirror the merged gzip fix: substitute 0 for out-of-range
values and coerce to int (RFC 1952), so floats and out-of-range system
clocks behave the same as in gzip and the non-streaming "w:gz" path
(which already delegates to GzipFile and was unaffected).
@harjothkhara harjothkhara changed the title gh-133998: Clamp out-of-range mtime in tarfile streaming gzip writes gh-133998: Fix tarfile error on out-of-range mtime in streaming gzip mode Jun 21, 2026
@harjothkhara

harjothkhara commented Jun 27, 2026 •

Copy link
Copy Markdown
Contributor Author

@serhiy-storchaka you flagged the same out-of-range mtime bug in tarfile._init_write_gz on gh-133998 (the gzip half landed in #134278). This PR applies the clamp-to-0 fix there, with tests and a NEWS entry. Can you please take a look when you have time?

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 90 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Oct 5, 2026
Comment thread Lib/test/test_tarfile.py Outdated

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for tests which cover all code paths. The Changelog entry describes well the fix. And tarfile behaves as gzip (same code in fact). So all good!

@vstinner vstinner added needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Oct 5, 2026
@vstinner
vstinner enabled auto-merge (squash) October 5, 2026 12:55
@vstinner

vstinner commented Oct 5, 2026

Copy link
Copy Markdown
Member

Oh, test_os.test_posix failed in the MSan job:

FAIL: test_fexecve (test.test_os.test_posix.PosixTester.test_fexecve)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Lib/test/test_os/test_posix.py", line 219, in test_fexecve
    support.wait_process(pid, exitcode=0)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/test/support/__init__.py", line 2491, in wait_process
    raise AssertionError(f"process {pid} exited with code {exitcode2}, "
                         f"but exit code {exitcode} is expected")
AssertionError: process 21288 exited with code 1, but exit code 0 is expected

MSan logs:

==> /home/runner/work/cpython/cpython/san_log.13546 <==
==13546==WARNING: MemorySanitizer failed to allocate 0x4000000000000021 bytes

==> /home/runner/work/cpython/cpython/san_log.21288 <==
execve failed, errno 2

Does MSan change excve() behavior?

cc @StanFromIreland

@vstinner

vstinner commented Oct 5, 2026

Copy link
Copy Markdown
Member

So far, I failed to reproduce the issue on Fedora 44 with clang version 22.1.8.

$ cat Modules/Setup.local
*disabled*
_bz2 _ctypes _curses _curses_panel _dbm _decimal _gdbm _hashlib
_lzma _sqlite3 _ssl _tkinter _uuid _zstd readline zlib

$ ./configure OPT="-O2 -g" --with-memory-sanitizer --with-assertions CC=clang LD=clang
$ make
$ ./python -m test -v test_os.test_posix 

The GHA CI uses CC.version: [clang] Ubuntu clang version 21.1.8 (6ubuntu1) on Ubuntu 26.04.1 LTS (Resolute Raccoon).

@vstinner

vstinner commented Oct 5, 2026

Copy link
Copy Markdown
Member

I created #158868 for the MSan failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting merge needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants