Skip to content

C API: Implement singletons as functions calls in the stable ABI 3.13 #115754

Closed
Closed
C API: Implement singletons as functions calls in the stable ABI 3.13#115754
Labels
topic-C-APItype-featureA feature request or enhancement

Description

@vstinner
vstinner
opened on Feb 21, 2024

Feature or enhancement

Currently, the C API implements many constants as macros. Examples:

#define Py_False _PyObject_CAST(&_Py_FalseStruct)
#define Py_None (&_Py_NoneStruct)

I propose to export important constants as symbols in the stable ABI and the limited C API, but still implement them as macros in the non-limited C API.

Linked PRs

Activity

  1. vstinner
    added
    type-featureA feature request or enhancement
    topic-C-API
    on Feb 21, 2024
  2. vstinner
    added 3 commits that reference this issue on Feb 21, 2024

    pythongh-115754: Export Py_None, Py_False, Py_True as symbols

    7c88eaf

    pythongh-115754: Export Py_None, Py_False, Py_True as symbols

    a5e6e52

    pythongh-115754: Export Py_None, Py_False, Py_True as symbols

    5dbd27b

  3. vstinner commented on Feb 21, 2024

    @vstinner
    vstinner
    on Feb 21, 2024
    MemberAuthor

    cc @encukou @davidhewitt @da-woods who might be interested by the topic.

  4. vstinner commented on Feb 21, 2024

    @vstinner
    vstinner
    on Feb 21, 2024
    MemberAuthor

    I also tried a different approach, use static const:

    diff --git a/Include/boolobject.h b/Include/boolobject.h
index 19aef5b1b8..a841bf1587 100644
--- a/Include/boolobject.h
+++ b/Include/boolobject.h
@@ -18,8 +18,8 @@ PyAPI_DATA(PyLongObject) _Py_FalseStruct;
 PyAPI_DATA(PyLongObject) _Py_TrueStruct;
 
 /* Use these macros */
-#define Py_False _PyObject_CAST(&_Py_FalseStruct)
-#define Py_True _PyObject_CAST(&_Py_TrueStruct)
+static PyObject * const Py_False = _PyObject_CAST(&_Py_FalseStruct);
+static PyObject * const Py_True = _PyObject_CAST(&_Py_TrueStruct);
 
 // Test if an object is the True singleton, the same as "x is True" in Python.
 PyAPI_FUNC(int) Py_IsTrue(PyObject *x);

    But it changes the constant type (make them constant in the limited C API), and it doesn't work with dlopen/dlsym.

    cc @pitrou

  5. vstinner commented on Feb 21, 2024

    @vstinner
    vstinner
    on Feb 21, 2024
    MemberAuthor

    I'm not sure how to categorize "important" vs "non-important" constants.

    Maybe only "pointers" to variables are interesting?

    Example of constants:

    #define PY_VECTORCALL_ARGUMENTS_OFFSET \
    (_Py_STATIC_CAST(size_t, 1) << (8 * sizeof(size_t) - 1))

#define Py_single_input 256   // compile.h

#define _PyDateTime_DATE_DATASIZE 4
#define PyDateTime_CAPSULE_NAME "datetime.datetime_CAPI"

#define Py_T_DOUBLE    4

#define S_IFMT 0170000   // fileutils.h

#define METH_VARARGS  0x0001

#define SEP L'\\'

#define Py_MATH_PI 3.14159265358979323846

#define Py_NAN ((double)NAN)

    In vim, I looked for :grep '\# *define [A-Za-z0-9_]\+ \+[^ \\]' Include/*.h.

  6. vstinner
    added a commit that references this issue on Feb 21, 2024

    pythongh-115754: Export Py_None, Py_False, Py_True as symbols

    614df03

  7. zooba commented on Feb 21, 2024

    @zooba
    zooba
    on Feb 21, 2024
    Member

    Can we export them as functions rather than symbols? Symbols are the reason we've struggled so hard to get the isolation we want, because we have to invent tricky solutions that wouldn't have been necessary had they been functions.

  8. zooba commented on Feb 21, 2024

    @zooba
    zooba
    on Feb 21, 2024
    Member

    And if functions, why not Py_GetConstant(int) and an enum to select which one is wanted? That's way more easily manageable from our POV (we'd want to document that the constant comes with an incremented refcount and is only valid in the active interpreter).

    I'm not sure how to categorize "important" vs "non-important" constants.

    My vote is for "important ones are PyObject *"

  9. vstinner
    added a commit that references this issue on Feb 21, 2024

    pythongh-115754: Export Py_None, Py_False, Py_True as symbols

    5002f4d

  10. vstinner commented on Feb 21, 2024

    @vstinner
    vstinner
    on Feb 21, 2024
    MemberAuthor

    Can we export them as functions rather than symbols?

    Sure, it's possible. But it would be slower and require C extensions to have a different API than the non-limited C API, no?

    I have a working implementation where Py_None and other constants are exported as symbols ("variables") and implemented as macros: best of both world.

  11. zooba commented on Feb 21, 2024

    @zooba
    zooba
    on Feb 21, 2024
    Member

    Well no, they could all use the same API if they want, it'd just be the function one ;-)

    And the stable ABI can be Py_GetConstant(PY_NONE) while the limited API includes #define Py_None Py_GetConstant(PY_NONE). It adds some overhead, obviously, but if we'd done it this way from the start then we wouldn't have the overhead on all the INCREF/DECREF calls instead (because we wouldn't have needed to immortalise static types and constants).

    Maybe we've run out of reasons that a function would be better than a data reference? I don't see any reason to assume that, and so I think having a function would be a safer ABI.

  12. vstinner commented on Feb 21, 2024

    @vstinner
    vstinner
    on Feb 21, 2024
    MemberAuthor

    Symbols are the reason we've struggled so hard to get the isolation we want

    Which isolation are you talking about?

    I don't get which problem you're trying to solve by using a function.

    #define Py_None Py_GetConstant(PY_NONE)

    Ah right, you would have the same API this way.

    But why would these specific constants need an indirection to pass through a function call whereas all other variables (PyAPI_DATA), such as Py_Version, are directly accessible?

  13. 30 remaining items

  14. NGRsoftlab
    added 2 commits that reference this issue on Apr 2, 2024

    Squashed commit of the following:

    1b5fe4c

    Squashed commit of the following:

    a016a54
  15. diegorusso
    added a commit that references this issue on Apr 17, 2024

    pythongh-115754: Add Py_GetConstant() function (python#116883)

    afed997
  16. vstinner
    added 10 commits that reference this issue on Oct 9, 2024

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR)

    0450bf0

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR)

    c4598d2

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES)

    66fe68e

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES)

    3d44519

    gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES) (#125195)

    6a39e96

    gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (#125194)

    b9a8ca0

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES)

    caad828

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES)

    038eb7a

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES)

    e65dcc8

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR)

    09bc803
  17. vstinner
    added a commit that references this issue on Oct 25, 2024

    gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (#125583)

    ebcc578
  18. ebonnal
    added a commit that references this issue on Jan 12, 2025

    pythongh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (python#12…

    da41ff3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-C-APItype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions