Feature or enhancement
Proposal:
The performance regression suite I run against CPython builds picked up a small slowdown in tempfile after #150477, which added the ValueError for a prefix or suffix with a directory component (gh-79459).
The check calls os.path.dirname() twice in _sanitize_params(), which runs on every mkstemp(), mkdtemp(), NamedTemporaryFile() and so on. A typical case is a function that processes a batch of uploaded documents and writes each one to a temporary file, because the library it hands them to only accepts a path:
for document in batch:
with tempfile.NamedTemporaryFile(suffix=".pdf") as f:
f.write(document)
f.flush()
extract_text(f.name)
On Lambda x86_64 (PGO+LTO), _sanitize_params() got about 56% slower, and creating 512 real files in /tmp about 2% slower. Locally it's roughly 375 ns more per call. Not a big deal next to the file system calls, but it's paid by every caller, including the ones that use the defaults.
I think it can be avoided without changing the behavior at all:
- Only validate values passed by the caller. The defaults (
template and an empty suffix) never contain a directory, so mkstemp() with no arguments doesn't need the check.
- For exact
str and bytes, check for the separator characters first, and only call os.path.dirname() when one is present. On POSIX that check is exact. On Windows, any value with \, / or : still goes through dirname(). Subclasses and path-like objects always go through dirname() as today.
With both, the no-argument case goes back to the cost from before #150477, and calls with an explicit prefix/suffix drop from about +380 ns to about +100 ns. I compared the fast path against posixpath.dirname() and ntpath.dirname() for every combination of the relevant characters up to 5 characters, as str and bytes, without any mismatch, and test_tempfile passes, including the refleak run.
Since this is a security fix, the goal is to keep exactly the same validation, just cheaper. Happy to send a PR. cc @vstinner
Feature or enhancement
Proposal:
The performance regression suite I run against CPython builds picked up a small slowdown in
tempfileafter #150477, which added theValueErrorfor aprefixorsuffixwith a directory component (gh-79459).The check calls
os.path.dirname()twice in_sanitize_params(), which runs on everymkstemp(),mkdtemp(),NamedTemporaryFile()and so on. A typical case is a function that processes a batch of uploaded documents and writes each one to a temporary file, because the library it hands them to only accepts a path:On Lambda x86_64 (PGO+LTO),
_sanitize_params()got about 56% slower, and creating 512 real files in/tmpabout 2% slower. Locally it's roughly 375 ns more per call. Not a big deal next to the file system calls, but it's paid by every caller, including the ones that use the defaults.I think it can be avoided without changing the behavior at all:
templateand an empty suffix) never contain a directory, somkstemp()with no arguments doesn't need the check.strandbytes, check for the separator characters first, and only callos.path.dirname()when one is present. On POSIX that check is exact. On Windows, any value with\,/or:still goes throughdirname(). Subclasses and path-like objects always go throughdirname()as today.With both, the no-argument case goes back to the cost from before #150477, and calls with an explicit
prefix/suffixdrop from about +380 ns to about +100 ns. I compared the fast path againstposixpath.dirname()andntpath.dirname()for every combination of the relevant characters up to 5 characters, asstrandbytes, without any mismatch, andtest_tempfilepasses, including the refleak run.Since this is a security fix, the goal is to keep exactly the same validation, just cheaper. Happy to send a PR. cc @vstinner