Repository navigation
Segfault freeing a deeply nested list after creating a JVM on the main thread (3.14 regression) #159639
Description
Bug report
Bug description:
After a JVM is created in-process on the main thread (as JPype and r5py do), freeing a deeply nested list segfaults on 3.14 and 3.15. The same script runs cleanly on 3.13.
This reproducer uses only ctypes and a JDK, with no third-party Python packages:
"""Free a deeply nested list after starting a JVM in-process (JNI via ctypes)."""
import ctypes
import pathlib
import subprocess
import sys
java_home = subprocess.run(
["java", "-XshowSettings:properties", "-version"], capture_output=True, text=True
).stderr.split("java.home = ")[1].split()[0]
libjvm = ctypes.CDLL(str(pathlib.Path(java_home, "lib", "server", "libjvm.so")))
class JavaVMOption(ctypes.Structure):
_fields_ = [("optionString", ctypes.c_char_p), ("extraInfo", ctypes.c_void_p)]
class JavaVMInitArgs(ctypes.Structure):
_fields_ = [
("version", ctypes.c_int),
("nOptions", ctypes.c_int),
("options", ctypes.POINTER(JavaVMOption)),
("ignoreUnrecognized", ctypes.c_ubyte),
]
flags = [arg.encode() for arg in sys.argv[1:]]
options = (JavaVMOption * max(len(flags), 1))(*[JavaVMOption(f, None) for f in flags])
args = JavaVMInitArgs(0x00010008, len(flags), options, 0) # JNI_VERSION_1_8
vm, env = ctypes.c_void_p(), ctypes.c_void_p()
assert libjvm.JNI_CreateJavaVM(ctypes.byref(vm), ctypes.byref(env), ctypes.byref(args)) == 0
print(sys.version.split()[0], "JVM started with", sys.argv[1:] or "default options", flush=True)
value = None
for _ in range(100_000):
value = [value]
print("built", flush=True)
del value
print("freed", flush=True)Results on Linux x86_64 (Ubuntu 24.04, glibc 2.39, OpenJDK 21.0.12, ulimit -s = 8192). The Python builds are the python-build-standalone interpreters installed by uv:
| Python | JVM options | Result |
|---|---|---|
| 3.13.14 | default | built, freed, exit 0 |
| 3.14.6 | default | built, then SIGSEGV (exit 139) |
| 3.15.0b2 | default | built, then SIGSEGV (exit 139) |
| 3.14.6 / 3.15.0b2 | -XX:+UnlockExperimentalVMOptions -XX:+DisablePrimordialThreadGuardPages |
built, freed, exit 0 |
Other observations:
- On 3.14.6, the crash happens when freeing the list. Nesting depths of 1,000 to 10,000 free cleanly; 30,000 and above crash.
- Without the JVM, 3.14 frees the 100,000-deep list without a problem.
- With the JVM loaded, unbounded Python recursion (
def f(): return f()) still raisesRecursionErrornormally. Only the deallocation path crashes. - No Python traceback or HotSpot
hs_errcrash log is produced. That fits the main thread's stack running into the JVM's guard pages.
My guess is that this comes from 3.14 checking C stack depth against the thread's actual stack bounds, which the trashcan deallocation path relies on. HotSpot adds guard pages to the primordial thread's stack, and CPython may not account for them. DisablePrimordialThreadGuardPages stops HotSpot from adding those guard pages, and the crash goes away. I haven't confirmed the mechanism in CPython's source.
We hit this in a pytest suite where some tests import r5py, which starts a JVM through JPype, and other tests build deeply nested values to check that the code under test doesn't recurse. The pytest-xdist workers died with "node down: Not properly terminated". As a workaround we now set JAVA_TOOL_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+DisablePrimordialThreadGuardPages".
CPython versions tested on:
3.13, 3.14, 3.15
Operating systems tested on:
Linux
Activity
NekoAsakura commentedon Oct 17, 2026
CPython 3.14 measures the thread's C stack once, when the thread starts, and from then on every recursion check just compares the stack pointer against that saved range (gh-91079). Since gh-124715 the trashcan in
_Py_Dealloc()uses the same saved range instead of counting depth. If the usable stack gets smaller afterwards, CPython keeps recursing past the real end. LoweringRLIMIT_STACKis enough to show it:3.13 frees the list. 3.14+ segfaults (3.14.7, 3.15.0rc2; Linux x86_64, glibc 2.39):
Not just deallocation:
repr(),==,pickle.dumps()andjson.dumps()on the same list segfault as well. Without thesetrlimitline they raiseRecursionError.cc: @markshannon @vstinner @encukou
corona10 commentedon Oct 19, 2026
It's reproducible on linux build but not in macOS.
encukou commentedon Oct 19, 2026
Could you try calling PyUnstable_ThreadState_SetStackProtection with a size that excludes the guard pages (if that size is known)?
vstinner commentedon Oct 19, 2026
I can reproduce @NekoAsakura's reproducer (#159639 (comment)) crash on the 3.14 branch, but I think that it's different from the original bug report. @NekoAsakura's reproducer calls
setrlimit(RLIMIT_STACK)which is another use case.When I reproduce the crash:
The stack size is exactly 1024 * 1024 bytes (
0x7ffffffff000 - 0x7fffffeff000is1048576bytes).When Python starts, it reads the stack size which is 8 MiB - 4 kiB. Example of gdb breakpoint on hardware_stack_limits():
So the problem is that PyThreadState members (c_stack_top, c_stack_soft_limit, and c_stack_hard_limit) are not recomputed when
setrlimit(RLIMIT_STACK)resizes the stack.NekoAsakura commentedon Oct 19, 2026
@encukou It works. Added after the
JNI_CreateJavaVMcall:With OpenJDK 21.0.12.1 on Linux x86_64:
Same on 3.14.7.
vstinner commentedon Oct 20, 2026
I can reproduce the crash using @ctriley's example (#159639 (comment)) on the Python 3.14 branch.
So when Python does crash, the stack size is 1008 kb (
0x7ffdb43db000 - 0x7ffdb42df000= 1032192 bytes). I'm not sure why Linux didn't grow to stack to the allowed limit of 8 MiB.gdb traces:
Python expects a stack of 8156 kb (
(0x7ffdb43da000 - 0x7ffdb3be3000) / 1024).Also, the current RLIMIT_STACK limit is 8192 kB:
vstinner commentedon Oct 20, 2026
Ah, I can also reproduce the jvm/ctypes crash on the main branch.
NekoAsakura commentedon Oct 20, 2026
@vstinner the jvm puts guard pages at 1 MiB:
Output: