Skip to content

gh-158893: Make os.strerror() thread-safe - #158927

Merged
vstinner merged 3 commits into
python:mainfrom
vstinner:strerror_r
Oct 6, 2026
Merged

gh-158893: Make os.strerror() thread-safe#158927
vstinner merged 3 commits into
python:mainfrom
vstinner:strerror_r

Conversation

@vstinner

@vstinner vstinner commented Oct 6, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

Make os.strerror() thread-safe: use the reentrant strerror_r() function if available.

  • The configure script now checks if strerror_r() is supported.
  • Add a stress test to test_free_threading.test_os (new module).
  • Add a comment on decode_current_locale() assertion which fails if the input string is mutated.
  • Add an assertion to _Py_DecodeLocale() to detect if the input string was mutated during the function call.

@vstinner
pythongh-158893: Make os.strerror() thread-safe
eb23db2 
Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.

* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
  the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
  string was mutated during the function call.
@bedevere-app bedevere-app Bot mentioned this pull request Oct 6, 2026
Closed
@vstinner

vstinner commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

The test depends on PR #158926. Without the second PR, the test is always skipped.

@vstinner

vstinner commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

Oh. Build fails on Android. It seems like Android C library (bionic) uses the GNU API for strerror_r().

  ../../Modules/posixmodule.c:13953:9: error: incompatible pointer to integer conversion initializing 'int' with an expression of type 'char * _Nonnull' [-Wint-conversion]
   13953 |     int len = strerror_r(code, small_buffer, buflen);
         |         ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ../../Modules/posixmodule.c:13973:17: error: incompatible pointer to integer conversion assigning to 'int' from 'char * _Nonnull' [-Wint-conversion]
   13973 |             len = strerror_r(code, buffer, buflen);
         |                 ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

@vstinner

vstinner commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

I clicked on [Update branch] to retrieve the sysconfig change, to run the test on Free Threaded builds.

@vstinner

vstinner commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

!buildbot FreeBSD15 NoGIL

@bedevere-bot

bedevere-bot commented Oct 6, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 7d18e20 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F158927%2Fmerge

The command will test the builders whose names match following regular expression: FreeBSD15 NoGIL

The builders matched are:

  • AMD64 FreeBSD15 NoGIL PR

@vstinner

vstinner commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

buildbot/AMD64 FreeBSD15 NoGIL PR

The build https://buildbot.python.org/#/builders/2369/builds/1 failed because of test_free_threading.test_capi which is unrelated to this change, and I wrote #158935 to fix this test.

The test passed successfully! 0:08:38 load avg: 4.04 mem: 453.2 MiB [345/559] test.test_free_threading.test_os passed.

I ran manually test_free_threading.test_os on the main branch on Free Threading on FreeBSD 15: the test crashed as expected. So the test covers well the modified code.

@vstinner
vstinner merged commit 6318730 into python:main Oct 6, 2026
56 of 57 checks passed
@vstinner
vstinner deleted the strerror_r branch October 6, 2026 21:02
@vstinner vstinner added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 6, 2026
@miss-islington-app

miss-islington-app Bot commented Oct 6, 2026

Copy link
Copy Markdown

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

@miss-islington-app

miss-islington-app Bot commented Oct 6, 2026

Copy link
Copy Markdown

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app

miss-islington-app Bot commented Oct 6, 2026

Copy link
Copy Markdown

Sorry, @vstinner, I could not cleanly backport this to 3.15 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker 6318730aeebdf7282fa9a18aca52af4e87c0c39d 3.15

@miss-islington-app

miss-islington-app Bot commented Oct 6, 2026

Copy link
Copy Markdown

Sorry, @vstinner, I could not cleanly backport this to 3.14 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker 6318730aeebdf7282fa9a18aca52af4e87c0c39d 3.14

@vstinner

vstinner commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

I will backport the fix to 3.14 and 3.15 branches once the 3.15 branch will be unblocked (next week). Right now, there are too many backports to 3.15 in the queue.

@vstinner

vstinner commented Oct 10, 2026

Copy link
Copy Markdown
Member Author

I created #159120 backport to 3.15 which combines multiple commits.

vstinner added a commit that referenced this pull request Oct 10, 2026
@vstinner
[3.15] gh-158893: Make os.strerror() thread-safe (#159120)
c774d15 
* gh-157695: Parse also _Py_HAVE_xxx variables in sysconfig (#158926)

Add parse_config_h() tests to test_sysconfig.

(cherry picked from commit 978a8be)

* gh-158893: Make os.strerror() thread-safe (#158927)

Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.

* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
  the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
  string was mutated during the function call.

* gh-158893: Add internal _Py_strerror() function (#158982)

Add a new internal _Py_strerror() function to Python/fileutils.c. It
uses strerror_r() if available, or use strerror() otherwise.

Replace all strerror(code) calls with _Py_strerror(code).

(cherry picked from commit 15dd735)
vstinner added a commit to vstinner/cpython that referenced this pull request Oct 10, 2026
@vstinner
[3.15] pythongh-158893: Make os.strerror() thread-safe (python#159120)
9559734 
* pythongh-157695: Parse also _Py_HAVE_xxx variables in sysconfig (python#158926)

Add parse_config_h() tests to test_sysconfig.

(cherry picked from commit 978a8be)

* pythongh-158893: Make os.strerror() thread-safe (python#158927)

Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.

* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
  the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
  string was mutated during the function call.

* pythongh-158893: Add internal _Py_strerror() function (python#158982)

Add a new internal _Py_strerror() function to Python/fileutils.c. It
uses strerror_r() if available, or use strerror() otherwise.

Replace all strerror(code) calls with _Py_strerror(code).

(cherry picked from commit 15dd735)
(cherry picked from commit c774d15)
@vstinner vstinner mentioned this pull request Oct 10, 2026
Merged
vstinner added a commit that referenced this pull request Oct 10, 2026
@vstinner
[3.14] gh-158893: Make os.strerror() thread-safe (#159120) (#159145)
b23d657 
[3.15] gh-158893: Make os.strerror() thread-safe (#159120)

* gh-157695: Parse also _Py_HAVE_xxx variables in sysconfig (#158926)

Add parse_config_h() tests to test_sysconfig.

(cherry picked from commit 978a8be)

* gh-158893: Make os.strerror() thread-safe (#158927)

Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.

* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
  the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
  string was mutated during the function call.

* gh-158893: Add internal _Py_strerror() function (#158982)

Add a new internal _Py_strerror() function to Python/fileutils.c. It
uses strerror_r() if available, or use strerror() otherwise.

Replace all strerror(code) calls with _Py_strerror(code).

(cherry picked from commit 15dd735)
(cherry picked from commit c774d15)
@hugovk hugovk removed needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner
@corona10 corona10 Awaiting requested review from corona10 corona10 is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping emmatyping is a code owner
@itamaro itamaro Awaiting requested review from itamaro itamaro is a code owner

Assignees

@vstinner vstinner

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vstinner @bedevere-bot @hugovk