Repository navigation
Removing old buffer support #85275
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/encukou'
closed_at = None
created_at = <Date 2020-06-24.13:43:42.640>
labels = ['expert-C-API', '3.10', '3.11']
title = 'Removing old buffer support'
updated_at = <Date 2021-08-05.08:33:02.859>
user = 'https://github.com/methane'bugs.python.org fields:
activity = <Date 2021-08-05.08:33:02.859>
actor = 'petr.viktorin'
assignee = 'petr.viktorin'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2020-06-24.13:43:42.640>
creator = 'methane'
dependencies = []
files = []
hgrepos = []
issue_num = 41103
keywords = ['patch']
message_count = 12.0
messages = ['372251', '372391', '381854', '381865', '381873', '398459', '398460', '398467', '398476', '398477', '398560', '398974']
nosy_count = 7.0
nosy_names = ['vstinner', 'petr.viktorin', 'methane', 'lukasz.langa', 'hroncok', 'miss-islington', 'tarun.johar']
pr_nums = ['21117', '27437', '27441']
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue41103'
versions = ['Python 3.10', 'Python 3.11']
Activity
methane commentedon Jun 24, 2020
https://docs.python.org/3/c-api/objbuffer.html
Old buffer protocol has been deprecated since Python 3.0.
It was useful to make transition from Python 2 easy.
But it's time to remove.
methane commentedon Jun 25, 2020
New changeset 6f8a6ee by Inada Naoki in branch 'master':
bpo-41103: Remove old buffer protocol support (bpo-21117)
6f8a6ee
hroncok commentedon Nov 25, 2020
I've just seen a fix of PyQt4 that basically copy pastes (some of) the removed code to PyQt4:
https://src.fedoraproject.org/rpms/PyQt4/pull-request/2#request_diff
What is the benefit of removing this? Is copy pasting the compatibility layer to (possibly many) different projects worth the "cleanup"?
In Fedora, at least 7 packages are broken so far:
PyQt4: https://bugzilla.redhat.com/show_bug.cgi?id=1895298
libsolv: https://bugzilla.redhat.com/show_bug.cgi?id=1896411
m2crypto: https://bugzilla.redhat.com/show_bug.cgi?id=1897148
apsw: https://bugzilla.redhat.com/show_bug.cgi?id=1897500
djvulibre: https://bugzilla.redhat.com/show_bug.cgi?id=1897558
wsaccel: https://bugzilla.redhat.com/show_bug.cgi?id=1899550
webassets: https://bugzilla.redhat.com/show_bug.cgi?id=1899555
vstinner commentedon Nov 25, 2020
PyObject_AsCharBuffer() is dangerous: it returns a dangling pointer by design. PyObject_GetBuffer() design is safer: the API ensures that the buffer remains valid until PyBuffer_Release() is called.
PyQt5 was updated to use the safe PyObject_GetBuffer()/PyBuffer_Release(). PyQt4 is no longer updated, that's why I proposed a downstream fix which copy/paste the old code. Changing PyQt4 to use the safe API was not worth it, since it's complex to redesign the impacted function qpycore_encode().
methane commentedon Nov 26, 2020
Thank you for reporting. I removed these APIs to get such feedback as early as possible.
We are free to revive these APIs if it breaks too much and some projects can not be fixed before Python 3.10 release.
Some project maintainers ignore deprecations and wait compile errors to fix soemthing. (e.g. rogerbinns/apsw#288 (comment))
That's why we need to break some projects during alpha phase.
Oh, no need to copy-paste compatibility layer for all projects. They can migrate to new APIs.
tarunjohar commentedon Jul 29, 2021
Also filed under https://bugs.python.org/issue44609
PEP-384 and PEP-652 define a stable ABI to be used with Python 3.2 and later. On Windows, symbols for the stable ABI are exported from the python3.dll shared library.
The following functions are present in Python 3.9 but have been removed from Python 3.10b3:
Without these functions, an extension cannot utilize the stable ABI to access the buffer memory of data structures. The buffer protocol is suggested as an alternative, but the buffer functions PyObject_GetBuffer() and PyBuffer_Release() are not present in the stable ABI.
While these two functions may be added to the stable ABI, removal of the four functions above makes Python 3.10 incompatible with previous versions. It is requested that the four functions be reinstated and maintained as described in PEP-652.
methane commentedon Jul 29, 2021
Oh, I didn't notice that APIs deprecated in Python 3.0 are included in stable ABIs defined at Python 3.2!
methane commentedon Jul 29, 2021
Should I resurrect only function implementation and symbol?
Or may I resurrect definitions in header files too?
15 remaining items
vstinner commentedon Jun 1, 2023
I closed my issue #105186 as a duplicate of this one. Copy of my message.
Would it be ok to remove the legacy "buffer" API in Python 3.13, scheduled for October 2024? I'm talking about functions:
This API is deprecated since Python 3.0 and is older than the new
Py_bufferAPI. ThePy_bufferAPI is safer: it has a PyBuffer_Release() method to notify when the consumer is done: the resource can be released. ThePy_bufferAPI is now commonly used.In June 2020, there was a first attempt to remove these functions in issue #85275: commit 6f8a6ee. It caused too many troubles and had to be reverted the following year (commit ce5e1a6, July 2021). The removed functions were still used by at least 6 projects:
The webassets project was also mentioned, but I'm not sure that its failure is related to these removal: see https://bugzilla.redhat.com/show_bug.cgi?id=1899555 I cannot find these functions in https://github.com/Kronuz/pyScss code right now.
Sadly, the 4 functions are part of the stable ABI and have to be kept there. I'm only asking to remove them in the C API. The issue #88775 was complaining that the functions were removed wheareas they are part of the stable ABI.
encukou commentedon Jun 1, 2023
Sorry I didn't get to this.
Removing them from public API, including the limited API, is AFAIK fine. But
Please keep (or add) tests for them, so they don't regress.
vstinner commentedon Jun 1, 2023
I propose to only remove them in the C API, but still export them in the stable ABI. That's what @methane's PR #105137 does.
gh-85275: Remove old buffer APIs (#105137)
Python 3.13 removed the old buffer interface
Python 3.13 removed the old buffer interface
Python 3.13 removed the old buffer interface
Python 3.13 removed the old buffer interface
Python 3.13 removed the old buffer interface