From 61aecd3e4ad43dd0639fbeceb0677b27fcb76017 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Mon, 15 Mar 2021 21:36:23 +0100 Subject: [PATCH 01/11] Add Include/README.rst --- Include/README.rst | 68 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 Include/README.rst diff --git a/Include/README.rst b/Include/README.rst new file mode 100644 index 00000000000000..aef4e51ca1859a --- /dev/null +++ b/Include/README.rst @@ -0,0 +1,68 @@ +The Python C API +================ + +The C API is divided into three sections: + +1. ``Include/`` +2. ``Include/cpython/`` +3. ``Include/internal/`` + + +Include: Limited API and Stable API +=================================== + +``Include/`` contains the public Limited API and the Stable API. The +Stable API is defined by :pep:`384`. + +Functions stealing references or returning borrowed references *must not* +be added to the Limited API API or the Stable API. A strong reference +*must* be returned. The API in ``Include/`` should not expose or leak +implementation details. + +Please start a public discussion before adding new C API to ``Include/``. + +Functions or macros with a ``_Py`` prefix do not belong in ``Include/``. + + +Include/cpython: CPython implementation details +=============================================== + +``Include/cpython/`` contains the public API that is excluded from the +Limited and Stable API. :pep:`384` introduced the ``Py_LIMITED_API`` +define to exclude functions from the Limited API. When a new API is +introduced, it has to be explicitly excluded using +``#ifndef Py_LIMITED_API``. If the author forgets it, the function is +added to be Stable and Limited API by mistake! + +Functions stealing references or returning borrowed references *must not* +be added to ``Include/cpython/``. A strong reference *must* be +returned. + +Please start a public discussion before adding new C API to +``Include/cpython/``. + +The API in ``Include/cpython/`` is prefixed with ``_Py``. + +See also `bpo-35134 `_. + + +Include/internal: The internal API +================================== + +With extern keyword +------------------- + +``Include/internal/`` with ``extern`` contains the C API which *must not +and can not* be used outside the CPython code base. Only stdlib +extensions built as built-in extensions can use it. + +In case of doubt, new C API must be added to the internal C API using +``extern``. + + +With PyAPI_FUNC or PyAPI_DATA +----------------------------- + +``Include/internal/`` with ``PyAPI_FUNC`` or ``PyAPI_DATA`` contains the +internal API which is exposed for specific use cases like debuggers and +profilers. From 28f9aa108e1749827c17b956eaac9a13d8bcf659 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Mon, 15 Mar 2021 23:48:44 +0100 Subject: [PATCH 02/11] Address review --- Include/README.rst | 41 ++++++++++++++++++----------------------- 1 file changed, 18 insertions(+), 23 deletions(-) diff --git a/Include/README.rst b/Include/README.rst index aef4e51ca1859a..478691f0f840a7 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -8,14 +8,14 @@ The C API is divided into three sections: 3. ``Include/internal/`` -Include: Limited API and Stable API +Include: Limited API and Stable ABI =================================== -``Include/`` contains the public Limited API and the Stable API. The -Stable API is defined by :pep:`384`. +``Include/`` contains the public Limited API and the Stable ABI. The +Stable ABI is defined by :pep:`384`. Functions stealing references or returning borrowed references *must not* -be added to the Limited API API or the Stable API. A strong reference +be added to the Limited API API or the Stable ABI. A strong reference *must* be returned. The API in ``Include/`` should not expose or leak implementation details. @@ -31,38 +31,33 @@ Include/cpython: CPython implementation details Limited and Stable API. :pep:`384` introduced the ``Py_LIMITED_API`` define to exclude functions from the Limited API. When a new API is introduced, it has to be explicitly excluded using -``#ifndef Py_LIMITED_API``. If the author forgets it, the function is -added to be Stable and Limited API by mistake! +``#ifndef Py_LIMITED_API``. Functions stealing references or returning borrowed references *must not* be added to ``Include/cpython/``. A strong reference *must* be returned. -Please start a public discussion before adding new C API to -``Include/cpython/``. - -The API in ``Include/cpython/`` is prefixed with ``_Py``. - See also `bpo-35134 `_. Include/internal: The internal API ================================== -With extern keyword -------------------- -``Include/internal/`` with ``extern`` contains the C API which *must not -and can not* be used outside the CPython code base. Only stdlib -extensions built as built-in extensions can use it. +With PyAPI_FUNC or PyAPI_DATA +----------------------------- -In case of doubt, new C API must be added to the internal C API using -``extern``. +``Include/internal/`` API defined with ``PyAPI_FUNC`` or ``PyAPI_DATA`` +contains the internal API which is exposed for specific use cases like +debuggers and profilers. -With PyAPI_FUNC or PyAPI_DATA ------------------------------ +With extern keyword +------------------- + +``Include/internal/`` API defined with ``extern`` *must not and can not* +be used outside the CPython code base. Only built-in stdlib extensions +(built with Py_BUILD_CORE_BUILTIN macro defined) can use it. -``Include/internal/`` with ``PyAPI_FUNC`` or ``PyAPI_DATA`` contains the -internal API which is exposed for specific use cases like debuggers and -profilers. +In case of doubt, new internal C API must be defined in the internal C +API with ``extern``. From ae6cbef35acb1cfd8b5661ea3d6e1063108426f3 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Tue, 16 Mar 2021 00:00:01 +0100 Subject: [PATCH 03/11] Fix one more stable API => ABI --- Include/README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Include/README.rst b/Include/README.rst index 478691f0f840a7..7cbe2697a5346d 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -28,7 +28,7 @@ Include/cpython: CPython implementation details =============================================== ``Include/cpython/`` contains the public API that is excluded from the -Limited and Stable API. :pep:`384` introduced the ``Py_LIMITED_API`` +Limited and Stable ABI. :pep:`384` introduced the ``Py_LIMITED_API`` define to exclude functions from the Limited API. When a new API is introduced, it has to be explicitly excluded using ``#ifndef Py_LIMITED_API``. From 1d8d351fec71d995c02555c15520a926b17ad8d0 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Tue, 16 Mar 2021 00:35:02 +0100 Subject: [PATCH 04/11] Remove duplicate word --- Include/README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Include/README.rst b/Include/README.rst index 7cbe2697a5346d..f391dc37973c7d 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -15,7 +15,7 @@ Include: Limited API and Stable ABI Stable ABI is defined by :pep:`384`. Functions stealing references or returning borrowed references *must not* -be added to the Limited API API or the Stable ABI. A strong reference +be added to the Limited API or the Stable ABI. A strong reference *must* be returned. The API in ``Include/`` should not expose or leak implementation details. From 70d5ceee4f8954b75d4afc577f8cfd9185041f9b Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Tue, 16 Mar 2021 00:48:12 +0100 Subject: [PATCH 05/11] Avoid using API when talking about a single function or macro --- Include/README.rst | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/Include/README.rst b/Include/README.rst index f391dc37973c7d..b025a324a15404 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -19,7 +19,8 @@ be added to the Limited API or the Stable ABI. A strong reference *must* be returned. The API in ``Include/`` should not expose or leak implementation details. -Please start a public discussion before adding new C API to ``Include/``. +Please start a public discussion before adding new functions or macros to +the Limited API. Functions or macros with a ``_Py`` prefix do not belong in ``Include/``. @@ -28,9 +29,9 @@ Include/cpython: CPython implementation details =============================================== ``Include/cpython/`` contains the public API that is excluded from the -Limited and Stable ABI. :pep:`384` introduced the ``Py_LIMITED_API`` -define to exclude functions from the Limited API. When a new API is -introduced, it has to be explicitly excluded using +Limited API and Stable ABI. :pep:`384` introduced the ``Py_LIMITED_API`` +define to exclude functions from the Limited API. When a new function or +macro is introduced, it has to be explicitly excluded using ``#ifndef Py_LIMITED_API``. Functions stealing references or returning borrowed references *must not* @@ -47,17 +48,18 @@ Include/internal: The internal API With PyAPI_FUNC or PyAPI_DATA ----------------------------- -``Include/internal/`` API defined with ``PyAPI_FUNC`` or ``PyAPI_DATA`` -contains the internal API which is exposed for specific use cases like -debuggers and profilers. +``Include/internal/`` functions or structures defined with ``PyAPI_FUNC`` +or ``PyAPI_DATA`` contains the internal API which is exposed for specific +use cases like debuggers and profilers. With extern keyword ------------------- -``Include/internal/`` API defined with ``extern`` *must not and can not* -be used outside the CPython code base. Only built-in stdlib extensions -(built with Py_BUILD_CORE_BUILTIN macro defined) can use it. +``Include/internal/`` function defined with ``extern`` *must not and can +not* be used outside the CPython code base. Only built-in stdlib +extensions (built with ``Py_BUILD_CORE_BUILTIN`` macro defined) can use +it. -In case of doubt, new internal C API must be defined in the internal C -API with ``extern``. +In case of doubt, new internal functions must be defined in the internal +C API with ``extern``. From 8b675c207b2a464a45c16036611f30c796802d32 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Tue, 16 Mar 2021 01:21:19 +0100 Subject: [PATCH 06/11] Address most of Guido's review --- Include/README.rst | 43 +++++++++++++++++++++---------------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/Include/README.rst b/Include/README.rst index b025a324a15404..2b46791569d90a 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -11,13 +11,14 @@ The C API is divided into three sections: Include: Limited API and Stable ABI =================================== -``Include/`` contains the public Limited API and the Stable ABI. The -Stable ABI is defined by :pep:`384`. +``Include/``, excluding the ``cpython`` and ``internal`` subdirectories, +contains the public Limited API and the Stable ABI. The Stable ABI is +defined by :pep:`384`. -Functions stealing references or returning borrowed references *must not* -be added to the Limited API or the Stable ABI. A strong reference -*must* be returned. The API in ``Include/`` should not expose or leak -implementation details. +*Note:* Functions stealing references or returning borrowed references +*must not* be added to the Limited API or the Stable ABI. A strong +reference *must* be returned. The API in ``Include/`` should not expose +or leak implementation details. Please start a public discussion before adding new functions or macros to the Limited API. @@ -29,16 +30,14 @@ Include/cpython: CPython implementation details =============================================== ``Include/cpython/`` contains the public API that is excluded from the -Limited API and Stable ABI. :pep:`384` introduced the ``Py_LIMITED_API`` -define to exclude functions from the Limited API. When a new function or -macro is introduced, it has to be explicitly excluded using -``#ifndef Py_LIMITED_API``. +Limited API and the Stable ABI. :pep:`384` introduced the +``Py_LIMITED_API`` define to exclude functions from the Limited API. +When a new function or macro is introduced, it has to be explicitly +excluded using ``#ifndef Py_LIMITED_API``. -Functions stealing references or returning borrowed references *must not* -be added to ``Include/cpython/``. A strong reference *must* be -returned. - -See also `bpo-35134 `_. +*Note:* Functions stealing references or returning borrowed references +*must not* be added to ``Include/cpython/``. A strong reference *must* +be returned. Include/internal: The internal API @@ -48,18 +47,18 @@ Include/internal: The internal API With PyAPI_FUNC or PyAPI_DATA ----------------------------- -``Include/internal/`` functions or structures defined with ``PyAPI_FUNC`` -or ``PyAPI_DATA`` contains the internal API which is exposed for specific -use cases like debuggers and profilers. +Functions or structures in ``Include/internal/`` defined with +``PyAPI_FUNC`` or ``PyAPI_DATA`` are internal functions which are +exposed only for specific use cases like debuggers and profilers. With extern keyword ------------------- -``Include/internal/`` function defined with ``extern`` *must not and can +Functions in ``Include/internal/`` defined with ``extern`` *must not and can not* be used outside the CPython code base. Only built-in stdlib extensions (built with ``Py_BUILD_CORE_BUILTIN`` macro defined) can use -it. +such functions. -In case of doubt, new internal functions must be defined in the internal -C API with ``extern``. +When in doubt, new internal C functions should be defined in +``Include/internal`` using ``extern``. From e419729248c4766564633fffd7eef20b91d51cd0 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Tue, 16 Mar 2021 09:50:59 +0100 Subject: [PATCH 07/11] Cleanup: - Improve Include/cpython section - Clarify Stable ABI/Limited API using wording from PEP 652 - Clean up requirements - Formatting --- Include/README.rst | 52 +++++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 24 deletions(-) diff --git a/Include/README.rst b/Include/README.rst index 2b46791569d90a..2136a129c89fe3 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -12,32 +12,36 @@ Include: Limited API and Stable ABI =================================== ``Include/``, excluding the ``cpython`` and ``internal`` subdirectories, -contains the public Limited API and the Stable ABI. The Stable ABI is -defined by :pep:`384`. +contains the public Limited API (Application Programming Interface) and +the Stable ABI (Application Binary Interface). :pep:`384` defined the +Stable ABI and the Limited API; two related, but distinct concepts. +The Stable ABI promises binary compatibility across subsequent versions of +CPython 3.x, given that the source code restricts itself to the Limited +API. The Limited API is a subset of the C API. -*Note:* Functions stealing references or returning borrowed references -*must not* be added to the Limited API or the Stable ABI. A strong -reference *must* be returned. The API in ``Include/`` should not expose -or leak implementation details. +Guidelines for expanding the Limited API: -Please start a public discussion before adding new functions or macros to -the Limited API. - -Functions or macros with a ``_Py`` prefix do not belong in ``Include/``. +- Functions *must not* steal references +- Functions *must not* return borrowed references +- Functions returning references *must* return a strong reference +- Macros should not expose implementation details +- Please start a public discussion before expanding the API +- Functions or macros with a ``_Py`` prefix do not belong in ``Include/``. Include/cpython: CPython implementation details =============================================== ``Include/cpython/`` contains the public API that is excluded from the -Limited API and the Stable ABI. :pep:`384` introduced the -``Py_LIMITED_API`` define to exclude functions from the Limited API. -When a new function or macro is introduced, it has to be explicitly -excluded using ``#ifndef Py_LIMITED_API``. +Limited API and the Stable ABI. When a new function or macro is +introduced, it has to be explicitly excluded from the Limited API using +``#ifndef Py_LIMITED_API``. + +Guidelines for expanding the public API: -*Note:* Functions stealing references or returning borrowed references -*must not* be added to ``Include/cpython/``. A strong reference *must* -be returned. +- Functions *must not* steal references +- Functions *must not* return borrowed references +- Functions returning references *must* return a strong reference Include/internal: The internal API @@ -52,13 +56,13 @@ Functions or structures in ``Include/internal/`` defined with exposed only for specific use cases like debuggers and profilers. -With extern keyword -------------------- +With the extern keyword +----------------------- -Functions in ``Include/internal/`` defined with ``extern`` *must not and can -not* be used outside the CPython code base. Only built-in stdlib -extensions (built with ``Py_BUILD_CORE_BUILTIN`` macro defined) can use -such functions. +Functions in ``Include/internal/`` defined with the ``extern`` keyword +*must not and can not* be used outside the CPython code base. Only +built-in stdlib extensions (built with the ``Py_BUILD_CORE_BUILTIN`` +macro defined) can use such functions. When in doubt, new internal C functions should be defined in -``Include/internal`` using ``extern``. +``Include/internal`` using the ``extern`` keyword. From ed2d7b27455874f084d950fb5c9a8843432a174f Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 17 Mar 2021 23:23:35 +0100 Subject: [PATCH 08/11] Don't mention the ABI --- Include/README.rst | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/Include/README.rst b/Include/README.rst index 2136a129c89fe3..1c72602833f5e9 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -8,16 +8,13 @@ The C API is divided into three sections: 3. ``Include/internal/`` -Include: Limited API and Stable ABI -=================================== +Include: Limited API +==================== ``Include/``, excluding the ``cpython`` and ``internal`` subdirectories, -contains the public Limited API (Application Programming Interface) and -the Stable ABI (Application Binary Interface). :pep:`384` defined the -Stable ABI and the Limited API; two related, but distinct concepts. -The Stable ABI promises binary compatibility across subsequent versions of -CPython 3.x, given that the source code restricts itself to the Limited -API. The Limited API is a subset of the C API. +contains the public Limited API (Application Programming Interface). +The Limited API is a subset of the C API, designed to guarantee ABI +stability across Python 3 versions, and is defined in :pep:`384`. Guidelines for expanding the Limited API: From e74ef86b37c14ba071dd1acb3366a4ae62051291 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 18 Mar 2021 00:14:42 +0100 Subject: [PATCH 09/11] Remove incorrect sentence regarding Py_LIMITED_API usage --- Include/README.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/Include/README.rst b/Include/README.rst index 1c72602833f5e9..739a9e0cba487c 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -30,9 +30,7 @@ Include/cpython: CPython implementation details =============================================== ``Include/cpython/`` contains the public API that is excluded from the -Limited API and the Stable ABI. When a new function or macro is -introduced, it has to be explicitly excluded from the Limited API using -``#ifndef Py_LIMITED_API``. +Limited API and the Stable ABI. Guidelines for expanding the public API: From 270e687d22275d1c2867648dfacb7497c71991c1 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 18 Mar 2021 21:14:03 +0100 Subject: [PATCH 10/11] How to add to the limited API from a given Py version --- Include/README.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Include/README.rst b/Include/README.rst index 739a9e0cba487c..308f5735771f2e 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -25,6 +25,11 @@ Guidelines for expanding the Limited API: - Please start a public discussion before expanding the API - Functions or macros with a ``_Py`` prefix do not belong in ``Include/``. +It is possible to add a function or macro to the Limited API from a +given Python version. For example, to add a function to the Limited API +from Python 3.10 and onwards, wrap it with +`#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000`. + Include/cpython: CPython implementation details =============================================== From 9dc55ed78f563f2c9a3c2da5f676a304cd5021e9 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 18 Mar 2021 21:17:42 +0100 Subject: [PATCH 11/11] Formatting --- Include/README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Include/README.rst b/Include/README.rst index 308f5735771f2e..d2467ca22fa038 100644 --- a/Include/README.rst +++ b/Include/README.rst @@ -28,7 +28,7 @@ Guidelines for expanding the Limited API: It is possible to add a function or macro to the Limited API from a given Python version. For example, to add a function to the Limited API from Python 3.10 and onwards, wrap it with -`#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000`. +``#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000``. Include/cpython: CPython implementation details