Skip to content

tempfile: speed up the prefix/suffix validation added in gh-79459 #159029

Description

@leandrodamascena

Feature or enhancement

Proposal:

The performance regression suite I run against CPython builds picked up a small slowdown in tempfile after #150477, which added the ValueError for a prefix or suffix with a directory component (gh-79459).

The check calls os.path.dirname() twice in _sanitize_params(), which runs on every mkstemp(), mkdtemp(), NamedTemporaryFile() and so on. A typical case is a function that processes a batch of uploaded documents and writes each one to a temporary file, because the library it hands them to only accepts a path:

for document in batch:
    with tempfile.NamedTemporaryFile(suffix=".pdf") as f:
        f.write(document)
        f.flush()
        extract_text(f.name)

On Lambda x86_64 (PGO+LTO), _sanitize_params() got about 56% slower, and creating 512 real files in /tmp about 2% slower. Locally it's roughly 375 ns more per call. Not a big deal next to the file system calls, but it's paid by every caller, including the ones that use the defaults.

I think it can be avoided without changing the behavior at all:

  • Only validate values passed by the caller. The defaults (template and an empty suffix) never contain a directory, so mkstemp() with no arguments doesn't need the check.
  • For exact str and bytes, check for the separator characters first, and only call os.path.dirname() when one is present. On POSIX that check is exact. On Windows, any value with \, / or : still goes through dirname(). Subclasses and path-like objects always go through dirname() as today.

With both, the no-argument case goes back to the cost from before #150477, and calls with an explicit prefix/suffix drop from about +380 ns to about +100 ns. I compared the fast path against posixpath.dirname() and ntpath.dirname() for every combination of the relevant characters up to 5 characters, as str and bytes, without any mismatch, and test_tempfile passes, including the refleak run.

Since this is a security fix, the goal is to keep exactly the same validation, just cheaper. Happy to send a PR. cc @vstinner

Activity

  1. added
    pendingThe issue will be closed if no feedback is provided
    on Oct 8, 2026
  2. vstinner commented on Oct 8, 2026

    @vstinner
    Member

    I don't think that it's worth it to micro-optimize tempfile._sanitize_params(). I prefer to keep the code simple and safe: always call dirname().

    creating 512 real files in /tmp about 2% slower.

    I wouldn't call "2% slower" a "performance regression".

    I suggest closing this issue.

  3. leandrodamascena commented on Oct 8, 2026

    @leandrodamascena
    Author

    Fair enough, I get the preference for keeping a security check as simple as possible. My thinking was mostly about high-volume workloads, like code that creates thousands of temporary files in a tight loop, where it adds up a bit. But you're right that it's small compared to the actual file creation, so I'm fine closing this. Thanks for taking the time to look at it!

  4. vstinner commented on Oct 8, 2026

    @vstinner
    Member

    My thinking was mostly about high-volume workloads, like code that creates thousands of temporary files in a tight loop

    I don't think that it's a realistic use case.

  5. leandrodamascena commented on Oct 8, 2026

    @leandrodamascena
    Author

    I don't think that it's a realistic use case.

    Makes sense, thanks! That's useful input for my tooling too. I'll raise the threshold it uses to flag a change as a regression, so smaller cases like this one get filtered out before they turn into a report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pendingThe issue will be closed if no feedback is provided

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions