gh-158886: Add macros of the limited C API to stable_abi.toml

[vstinner]

• Issue: [C API] Add all macros to Misc/stable_abi.toml #158886

[vstinner]

The Doctest job fails with:

sphinx.errors.SphinxParallelError: sphinx.errors.ExtensionError: Handler <function add_annotations at 0x7af8150d5fd0> for event 'doctree-read' threw an exception (exception: Object type mismatch in limited API annotation for PyObject_New: 'function' != 'macro')

I documented PyObject_New() as a function in stable_abi.toml ([function.PyObject_New]) to fix to fix Doctest, but it doesn't work as expected.

Problem: macros declared as functions by stable_abi.toml are checked by Lib/test/test_stable_abi_ctypes.py which fails since they are macros and not functions...

[vstinner]

I updated the PR to document all macros as macros, not as functions.

[vstinner]

Updated Doctest error:

sphinx.errors.SphinxParallelError: sphinx.errors.ExtensionError: Handler <function add_annotations at 0x7353c3fd9fd0> for event 'doctree-read' threw an exception (exception: Object type mismatch in limited API annotation for PyImport_ImportModuleEx: 'macro' != 'function')

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34957345 | 📁 Comparing 1db4cb9 against main (eab2d16)

  🔍 Preview build  

33 files changed · ± 33 modified

± Modified

• c-api/allocation.html
• c-api/bool.html
• c-api/bytearray.html
• c-api/bytes.html
• c-api/capsule.html
• c-api/complex.html
• c-api/curses.html
• c-api/dict.html
• c-api/exceptions.html
• c-api/float.html
• and 23 more...

[vstinner]

I updated Doc/tools/extensions/c_annotations.py to accept that some documented functions are defined as macros by stable_abi.toml.

[encukou]

Could you limit this PR to the refcounting macros, and keep the issue open for the rest?
I don't think they should all be added at once.

---

One problem is that this annotates macros as “Part of the stable ABI”:

But, macros are not ABI.

We already have this issue with, for example, Py_BEGIN_ALLOW_THREADS, but there it's at least clear from the prose (or context) that it's a macro. Adding this to things documented as functions would be misleading.

---

For macros like Py_ULL or PyAPI_DATA, I think it would be better to treat them as exposed by mistake, rather than limited API. For ones like Py_IS_FINITE, we probably need more discussion (or not -- we can leave them in a gray area and focus on other stuff).

For macros like PyBytes_Check, IMO we should document the (approximate) expansion, so wrappers like PyO3 can reimplement them using just the stable ABI.

[vstinner]

Could you limit this PR to the refcounting macros, and keep the issue open for the rest?

I created #158941 to add some macros.

I don't think they should all be added at once.

Ok.

One problem is that this annotates macros as “Part of the stable ABI”

For end users (developers using of the limited CAPI), I don't think that it makes a big difference if a function is implemented as a macro or as a function. In the limited C API, it's the same.

I like how some macros are documented as function to show parameter types and return type, that's nice!

For macros like Py_ULL or PyAPI_DATA, I think it would be better to treat them as exposed by mistake, rather than limited API.

I think that it's fine to not document some special macros, but Misc/stable_abi.toml should be complete since it's used by tools to check if a C extension uses correctly the limited C API / stable ABI, no?

Note: https://docs.python.org/dev/c-api/intro.html#c.PyAPI_DATA and https://docs.python.org/dev/c-api/intro.html#c.Py_ULL are documented.

For ones like Py_IS_FINITE, we probably need more discussion (or not -- we can leave them in a gray area and focus on other stuff).

Py_IS_FINITE() is a deprecated alias to isfinite():

// Py_IS_FINITE(X)
// Return 1 if float or double arg is neither infinite nor NAN, else 0.
// Soft deprecated since Python 3.14, use isfinite() instead.
#define Py_IS_FINITE(X) isfinite(X)

It's documented: https://docs.python.org/dev/c-api/float.html#c.Py_IS_FINITE.

What should we not add it to stable_abi.toml? It's just a fact that it's part of the limited C API.

For macros like PyBytes_Check, IMO we should document the (approximate) expansion, so wrappers like PyO3 can reimplement them using just the stable ABI.

Ok, later I will prepare a PR focused on Check functions.

[vstinner]

I mark this PR as a draft. I will split it into smaller PRs, as suggested by @encukou.

[encukou]

For end users (developers using of the limited CAPI), I don't think that it makes a big difference if a function is implemented as a macro or as a function. In the limited C API, it's the same.

That's true! But there are also users of the Stable ABI that don't use C, such as (our) ctypes.pythonapi or (practical) PyO3.
There, the distinction between macros and actual functions matters a lot.
(But on the other hand, static inline functions are pretty much the same as macros.)

I like how some macros are documented as function to show parameter types and return type, that's nice!

Yes! I just want to be clear that they're not in the ABI :)

What should we not add it to stable_abi.toml? It's just a fact that it's part of the limited C API.

No. PEP 652 specifically says about the manifest (which is now named Misc/stable_abi.toml):

The manifest will also serve as the definitive list of the Limited API.

Discrepancies between the TOML file and what's actually exposed are bugs. They might be bugs in the manifest, but they might also be bugs in the headers.
(Also, the remaining ones are probably low-priority bugs.)

The manifest was initially generated from PC/python3dll.c so the ABI list is ~complete, but macros are added manually so a bunch of them is missing.

Ok, later I will prepare a PR focused on Check functions.

Since these are all the same, it might make sense to add one common description and link it everywhere.
Do you want to leave this to me?

[vstinner]

Since these are all the same, it might make sense to add one common description and link it everywhere. Do you want to leave this to me?

Since you have an idea on how these Check functions should be document, oh sure, go ahead!

This PR should contain all Check functions which are part of the limited C API and are not documented in stable_abi.toml yet. So you can check the list.

[encukou]

Hmm, turns out I already started on that, for capi-workgroup/decisions#115 :)