3.14: objects de-immortalized by old stable-ABI extensions break truthiness checks in specialized bytecode

[clairernovotny]

Bug report

Bug description:

Extensions compiled against 3.11 or older headers implement Py_DECREF as an inline --ob_refcnt. That includes abi3 wheels built on 3.11, which are common. Since 3.12 the interpreter hands out references to immortal objects without incrementing their refcount, so each such reference the extension releases is a net decrement. 3.14 starts immortal objects at 3 << 30 and treats (int32_t)ob_refcnt < 0 as immortal, which leaves a margin of 2**30 (see gh-125174).

That margin runs out in real long-running processes. Home Assistant on 3.14.6 crashes after hours to days because PyAV's cp311-abi3 wheels drain True by a few counts per av.open() and per FFmpeg log line (PyAV-Org/PyAV#2420, home-assistant/core#178218). A core dump from an affected instance had _Py_TrueStruct.ob_refcnt == 0x7ffb2327, which is 2**30 + 318,681 below 0xC0000000.

On 3.12 and 3.13, losing immortality is fairly benign: the object is then refcounted like any other, and results stay correct. On 3.14 it is a silent correctness failure. The reason is in the default build's stackrefs:

• PyStackRef_IsTrue/IsFalse/IsNone compare the tagged bits exactly.
• _PyStackRef_FromPyObjectNew and PyStackRef_MakeHeapSafe decide whether to tag with _Py_IsImmortal(), which reads the refcount.
• PyStackRef_FromPyObjectSteal decides with ob_flags.

Once True's refcount is below 2**31, a True loaded by a specialized attribute load (LOAD_ATTR_INSTANCE_VALUE and others) or passed through RETURN_VALUE is untagged, and POP_JUMP_IF_TRUE treats it as false. True from constants, comparisons and C calls still tests true, so the failures look random. In Home Assistant, Thread.name asserted Thread.__init__() not called, Condition.notify raised cannot notify on un-acquired lock, traceback printed None as the exception type, and so on, across threads, until the process exited.

Reproducer. ctypes stands in for the extension's unbalanced decrefs:

import ctypes, sys

print(sys.version)

class C:
    def __init__(self):
        self.flag = True

def check_attr(o):
    if not o.flag:                      # LOAD_ATTR_INSTANCE_VALUE, POP_JUMP_IF_TRUE
        return "WRONG: o.flag is True but tested false"
    return "ok"

def get_true():
    return True

def check_return():
    if not get_true():                  # CALL_PY_EXACT_ARGS, RETURN_VALUE, POP_JUMP_IF_TRUE
        return "WRONG: get_true() returned True but tested false"
    return "ok"

o = C()
for _ in range(1000):                   # warm up so both functions get specialized
    check_attr(o), check_return()

refcnt = ctypes.c_uint32.from_address(id(True))   # low 32 bits of ob_refcnt (64-bit little-endian)
print("before: ob_refcnt = %#x" % refcnt.value, check_attr(o), check_return())
refcnt.value = 2**31 - 10**6            # 3.14: 2**30 + 10**6 raw decrements from 0xc0000000
print("after:  ob_refcnt = %#x" % refcnt.value, check_attr(o), check_return())
print("o.flag is True:", o.flag is True, "| bool(o.flag):", bool(o.flag))

Output with the official Docker images (python:3.x, Linux aarch64, default build):

3.14.7 (main, Sep 19 2026, 03:34:22) [GCC 14.2.0]
before: ob_refcnt = 0xc0000000 ok ok
after:  ob_refcnt = 0x7ff0bdc0 WRONG: o.flag is True but tested false WRONG: get_true() returned True but tested false
o.flag is True: True | bool(o.flag): True

3.15.0rc2 (main, Sep 19 2026, 03:32:36) [GCC 14.2.0]
before: ob_refcnt = 0xc0000000 ok ok
after:  ob_refcnt = 0x7ff0bdc0 WRONG: o.flag is True but tested false WRONG: get_true() returned True but tested false
o.flag is True: True | bool(o.flag): True

3.13.15 (main, Sep 19 2026, 03:38:46) [GCC 14.2.0]
before: ob_refcnt = 0xffffffff ok ok
after:  ob_refcnt = 0x7ff0bdc0 ok ok
o.flag is True: True | bool(o.flag): True

3.12.14 (main, Sep 19 2026, 03:37:20) [GCC 14.2.0]
before: ob_refcnt = 0xffffffff ok ok
after:  ob_refcnt = 0x7ff0bdc0 ok ok
o.flag is True: True | bool(o.flag): True

The value has to be far enough below 2**31. At exactly 0x7fffffff, the first mortal-path incref brings True back to 0x80000000, and it is immortal again.

The drift itself is the extension's bug, and PyAV is fixing its build. But 3.14 turns it from a lost optimization into wrong branches, and nothing in the process points at the cause. Some options, in no particular order:

• Tag consistently. _PyStackRef_FromPyObjectNew and PyStackRef_MakeHeapSafe could use ob_flags & _Py_IMMORTAL_FLAGS as PyStackRef_FromPyObjectSteal does. Alternatively, PyStackRef_IsTrue/IsFalse/IsNone could ignore the tag bit. Either would fix the truth tests, though not the drift.
• Treat statically allocated objects (_Py_STATICALLY_ALLOCATED_FLAG) as immortal whatever their refcount says.
• Periodically reset the refcount of static immortal singletons, for example during GC, as suggested in "Immortal" objects aren't immortal and that breaks things. #125174.

CPython versions tested on:

3.12, 3.13, 3.14, 3.15

Operating systems tested on:

Linux

[picnixz]

should I understand that the problem is extensions not properly managing refcounts or is there something else? I don't understand the issue exactly here. AFAIU:

The drift itself is the extension's bug, and PyAV is fixing its build

So that's the extension problem? or is it ours?