You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the sys.abiflags attribute is only set on Unix systems. On Windows, there are no sys.abiflags set (i.e. hasattr(sys, 'abiflags') -> False). It is hard to tell if Python is in free-threaded or debug builds. Although developers can use something like sysconfig.get_config_var('Py_GIL_DISABLED'). It is not well documented and is not convenient.
It would be nice if we could set ABIFLAGS in pyconfig.h and enable sys.abiflags on Windows. This would simplify much code in the wild world and also in the CPython repo.
I'm not convinced this is non-impactful elsewhere. We would need to make sure that anywhere that uses something "defined" by sys.abiflags is actually using those flags (e.g. the debug flag doesn't currently appear in ABI tags, and so we may be breaking the definition by putting this attribute here).
The lack of documentation for experimental builds is somewhat deliberate right now. It's being documented outside of the main docs, as we don't want to confuse users with features that are not officially part of CPython. If you want more info about how to work with free-threaded builds, you'll have to find that (I don't remember where it is, but the discussion forums are a good place to start looking).
I'm not convinced this is non-impactful elsewhere. We would need to make sure that anywhere that uses something "defined" by sys.abiflags is actually using those flags
Here is the search result of the following statements on GitHub:
There are a number of differences like this between Windows and other platforms. It's really meaningless to test any of this on non-Windows and then assume it applies cross-platform, it really won't. (Also don't put too much faith in the docs. They're largely written by Linux developers for Linux developers, and only rarely branch out into other platforms.)
I wonder if adding something that did not exist before would break historical things. EXT_SUFFIX is calculated in C here which does not rely on ABIFLAGS (not exists):
I wonder if adding something that did not exist before would break historical things.
Yes, this is my concern, but I'm far more concerned about 3rd party code using sys.abiflags to calculate things that currently don't depend on ABIFLAGS and won't also be updated.
This seems much like the discussion we had about using cp3.10 instead of cp310, where eventually we decided to not make the change because it would break practically everyone who made any assumption about the tag at all for limited benefit.
Being able to add t here would be an actual benefit, but I'm not sure it's worth the risk of breakage for 2-3 versions (when it should go away again when we go back to a single ABI). If there are other ABI flags worth adding that would be permanent, we should consider it, but I'm not aware of any right now.
Being able to add t here would be an actual benefit, but I'm not sure it's worth the risk of breakage for 2-3 versions (when it should go away again when we go back to a single ABI). If there are other ABI flags worth adding that would be permanent, we should consider it, but I'm not aware of any right now.
sys.abiflags has been around since 3.2 and was empty from 3.8-3.12. It doesn't seem to be a problem to leave it as an empty string for years on end, and it provides a standard place to make a mark if there is ever another need for multiple ABIs.
For a concrete use-case, I'm trying to patch tox-gh-actions to detect free-threaded builds (ymyzk/tox-gh-actions#199), which need slightly different configurations from the regular GIL builds. It's breaking for Windows, and falling back to 3.13 for Windows just means I can't test free-threaded Python on Windows unless I find some other contortion. For the moment, I will probably parse EXT_SUFFIX, but that hardly seems less brittle than a sequence of character codes that may be empty.
sys.abiflags has been around since 3.2 and was empty from 3.8-3.12
On Windows, it has been absent for all of that time. Changing from AttributeError to a meaningful value is a significant change for existing code. We can only hope that they all assume that "absent" implies "empty string".
For a concrete use-case, I'm trying to patch tox-gh-actions to detect free-threaded builds
Does sysconfig.get_config_var("Py_GIL_DISABLED") == 1 not suffice?
We can only hope that they all assume that "absent" implies "empty string".
For Python 3.8-3.12, it is true for normal builds. However, for Python 3.13, we have two different builds (the normal build and the free-threading build). It is inappropriate to assume the "absent" implies "empty string" in Python 3.13 and onwards.
I'm talking about backwards compatibility, and how programs will change behaviour if we change our behaviour. As I've said a couple of times, that is my primary concern here.
If you don't have any reason why it's safe, then let's deprecate sys.abiflags being missing on Windows and in 3.16 we can add it back.
Until then, use sysconfig to find out if CPython was built with free-threading enabled.
Yeah, basically. The docs explicitly say it's only available on Unix, so it would get updated to indicate that it may be set on all platforms, and probably specify that a missing attribute is intended to be interpreted as an empty string. It may also need clarification that it isn't necessarily used the same on all platforms (it shouldn't be used to construct extension module suffixes, for example).
In Porting Notes for the next two releases we'd say that Windows is going to start setting it to an empty string rather than being absent in 3.16. No reason to actually use the word "deprecation" except to explain what process we're following for changing a user-visible and documented API.
The relevance of abi_features here is whether it removes the need to provide abiflags on Windows at all. Right now I get the impression that it would, especially since you can't just use sys.abiflags on Windows the same way you would on other platforms. Having a different way to get the desired information would be preferable.
The benefit of such a change would still be years in the future, at which point I'd hope that sys.abi_features would be a better alternative.
The sys.abi_features alternative does not replace the incoming change of sys.abiflags and vice versa. Both can benefit the community independently.
The recent update of the action/setup-python adds 3.13t support (release note). I changed the code to determine PYTHON_TAG which is used in the artifact name (e.g., coverage-${{ env.PYTHON_TAG }}-${{ runner.os }}.xml) in metaopt/optree#198 (files).
This would benefit from a valid sys.abiflags on all platforms.
The incoming sys.abi_features would be useful. But in scripting and CIs, a good sys.abiflags can make the inline code shorter and easier to maintain.
The addition for sys.abi_features in PEP 780 does not imply the retirement of sys.abiflags. Although a sys.abi_features can resolve the issue as this one is titled "make check free-threaded/debug builds easier on Windows".
A single sys.abiflags variable can be used more easily than ... if ... else ... statements:
Each time we add a new build that differs in API, we will need to update this definition.
Back to the topic of the absence of sys.abiflags on Windows. There are many places using hasattr(sys, 'abiflags') and getattr(sys, 'abiflags', ''). Sometimes, developers forget about this difference of presence between Windows and UNIX system. For example:
build-details.json is an installation-wide file, meaning that it MUST only contain information that is constant across all environments of the Python installation.
It should work on all platforms, including Windows.
If we deprecate sys.abiflags at all, we will have to maintain the following:
(
'{installed_platbase}/include/{implementation_lower}{py_version_short}' % (
installed_platbase,
implementation_lower,
py_version_short,
)
+ ('t' if 'free-threading' in sys.abi_features else '')
+ ('d' if 'debug' in sys.abi_features else '')
+ ('x' if 'some-feature-in-future' in sys.abi_features else '')
)
There should be a sysconfig entry for this path. Nobody should be constructing it by hand, certainly not from sys properties (from sysconfig properties, potentially). sysconfig.get_config_var("ABIFLAGS") isn't going anywhere.
And sysconfig can use whatever logic or variables it likes, because it's internal. The whole point of the module is to pull together our mess of internal settings and present a stable, easy-to-use interface for users. When users ignore it, they will find they have to work too hard, and if they ignore us when we tell them there's an easier way, there's nothing more we can do for them.
This could just flat out do with better logic. {0.name[0]}p is a ridiculous way to get cp or pp (or ip or jp(?) or...?) - they probably want the table from packaging.tags. sysconfig.get_config_var("abi_thread") will give the optional t already, and will presumably go back to being empty when there's no longer an ABI difference. (I don't actually know where that variable came from - I didn't add it or approve it - but it's there.)
What we probably need is for sysconfig._init_non_posix to add an ABIFLAGS value that matches abi_thread (we don't want to touch abiflags lowercase, because that is used to calculate paths). ABIFLAGS has always been empty on Windows, AFAIK, but if it's being used in a cross-platform way already, it should be being used in the right places to generate correct filenames (though more likely not at all, since EXT_SUFFIX suffices for extension modules and nowhere else would've matched a POSIX name anyway since we rarely use the same name formats).
This could just flat out do with better logic. {0.name[0]}p is a ridiculous way to get cp or pp (or ip or jp(?) or...?) - they probably want the table from packaging.tags.
I just want to get a hash key for the Python build which is used in the CI artifact name. It is not a formal use case for the tags. And my package only officially supports CPython and PyPy.
sysconfig.get_config_var("abi_thread") will give the optional t already, and will presumably go back to being empty when there's no longer an ABI difference.
This is not documented.
What we probably need is for sysconfig._init_non_posix to add an ABIFLAGS value that matches abi_thread (we don't want to touch abiflags lowercase, because that is used to calculate paths).
I just want to get a hash key for the Python build which is used in the CI artifact name.
Why not just use the full implementation name then? I assumed this had some user-visible aspect (e.g. what a user types to install a particular runtime version).
This is not documented.
Yeah, again, I wasn't involved in adding that one. Whoever was clearly forgot about whether this needed to be publicly accessible. I probably would've argued for just adding ABIFLAGS, so thanks for doing that!
changed the title [-]Set `sys.abiflags` on Windows to make check free-threaded/debug builds easier[/-][+]Set `sys.abiflags` on Windows[/+]on Apr 11, 2025
Activity
zooba commentedon Dec 2, 2024
I'm not convinced this is non-impactful elsewhere. We would need to make sure that anywhere that uses something "defined" by
sys.abiflagsis actually using those flags (e.g. the debug flag doesn't currently appear in ABI tags, and so we may be breaking the definition by putting this attribute here).The lack of documentation for experimental builds is somewhat deliberate right now. It's being documented outside of the main docs, as we don't want to confuse users with features that are not officially part of CPython. If you want more info about how to work with free-threaded builds, you'll have to find that (I don't remember where it is, but the discussion forums are a good place to start looking).
XuehaiPan commentedon Dec 2, 2024
Here is the search result of the following statements on GitHub:
getattr(sys, 'abiflags', ''): 5.8k matches https://github.com/search?q=getattr%28sys%2C+%27abiflags%27%2C+%27%27%29&type=codegetattr(sys, "abiflags", ""): ~200 matches https://github.com/search?q=getattr%28sys%2C+%22abiflags%22%2C+%22%22%29&type=codeIssue on the
cpythonGitHub repo: Cross compiling issue on WindowsThe debug flag is included in the ABI tags. Here is the result that the Python is configured with
--with-pydebug.dis included in theabiflagsfor debug builds: https://docs.python.org/3/using/configure.html#python-debug-buildA fresh installation built with
--with-pydebug --disable-gil:zooba commentedon Dec 4, 2024
Not on Windows - we use a
_dsuffix on the name rather than putting it in the ABI (historical reasons, but not easily changed).There are a number of differences like this between Windows and other platforms. It's really meaningless to test any of this on non-Windows and then assume it applies cross-platform, it really won't. (Also don't put too much faith in the docs. They're largely written by Linux developers for Linux developers, and only rarely branch out into other platforms.)
XuehaiPan commentedon Dec 5, 2024
There are no
ABIFLAGSin C andsys.abiflagsin Python on Windows.I wonder if adding something that did not exist before would break historical things.
EXT_SUFFIXis calculated in C here which does not rely onABIFLAGS(not exists):cpython/Include/internal/pycore_importdl.h
Lines 106 to 133 in 6cf7794
zooba commentedon Dec 5, 2024
Yes, this is my concern, but I'm far more concerned about 3rd party code using
sys.abiflagsto calculate things that currently don't depend onABIFLAGSand won't also be updated.This seems much like the discussion we had about using
cp3.10instead ofcp310, where eventually we decided to not make the change because it would break practically everyone who made any assumption about the tag at all for limited benefit.Being able to add
there would be an actual benefit, but I'm not sure it's worth the risk of breakage for 2-3 versions (when it should go away again when we go back to a single ABI). If there are other ABI flags worth adding that would be permanent, we should consider it, but I'm not aware of any right now.effigies commentedon Dec 8, 2024
sys.abiflagshas been around since 3.2 and was empty from 3.8-3.12. It doesn't seem to be a problem to leave it as an empty string for years on end, and it provides a standard place to make a mark if there is ever another need for multiple ABIs.For a concrete use-case, I'm trying to patch
tox-gh-actionsto detect free-threaded builds (ymyzk/tox-gh-actions#199), which need slightly different configurations from the regular GIL builds. It's breaking for Windows, and falling back to 3.13 for Windows just means I can't test free-threaded Python on Windows unless I find some other contortion. For the moment, I will probably parseEXT_SUFFIX, but that hardly seems less brittle than a sequence of character codes that may be empty.zooba commentedon Dec 9, 2024
On Windows, it has been absent for all of that time. Changing from
AttributeErrorto a meaningful value is a significant change for existing code. We can only hope that they all assume that "absent" implies "empty string".Does
sysconfig.get_config_var("Py_GIL_DISABLED") == 1not suffice?XuehaiPan commentedon Dec 9, 2024
For Python 3.8-3.12, it is true for normal builds. However, for Python 3.13, we have two different builds (the normal build and the free-threading build). It is inappropriate to assume the "absent" implies "empty string" in Python 3.13 and onwards.
zooba commentedon Dec 10, 2024
I'm talking about backwards compatibility, and how programs will change behaviour if we change our behaviour. As I've said a couple of times, that is my primary concern here.
If you don't have any reason why it's safe, then let's deprecate
sys.abiflagsbeing missing on Windows and in 3.16 we can add it back.Until then, use
sysconfigto find out if CPython was built with free-threading enabled.effigies commentedon Dec 10, 2024
I'm curious what this would mean in practice. Is this just a note in the docs?
zooba commentedon Dec 10, 2024
Yeah, basically. The docs explicitly say it's only available on Unix, so it would get updated to indicate that it may be set on all platforms, and probably specify that a missing attribute is intended to be interpreted as an empty string. It may also need clarification that it isn't necessarily used the same on all platforms (it shouldn't be used to construct extension module suffixes, for example).
In Porting Notes for the next two releases we'd say that Windows is going to start setting it to an empty string rather than being absent in 3.16. No reason to actually use the word "deprecation" except to explain what process we're following for changing a user-visible and documented API.
XuehaiPan commentedon Dec 11, 2024
Sounds reasonable to me.
1 remaining item
AA-Turner commentedon Mar 25, 2025
See also PEP 780, which might be a better long-term solution?
A
XuehaiPan commentedon Mar 25, 2025
It seems the PEP 780 does not exist on https://peps.python.org/pep-0780 also nor on https://github.com/python/peps/tree/main/peps.
UPDATE: Here it is:
merwok commentedon Mar 26, 2025
Steve (I think) suggested adding abi_features to sys.implementation
Would that be clearer than un-hiding sys.abiflags?
zooba commentedon Mar 26, 2025
The relevance of
abi_featureshere is whether it removes the need to provideabiflagson Windows at all. Right now I get the impression that it would, especially since you can't just usesys.abiflagson Windows the same way you would on other platforms. Having a different way to get the desired information would be preferable.XuehaiPan commentedon Mar 27, 2025
Just repost my comment in #131717 (comment).
The addition for
sys.abi_featuresin PEP 780 does not imply the retirement ofsys.abiflags. Although asys.abi_featurescan resolve the issue as this one is titled "make check free-threaded/debug builds easier on Windows".A single
sys.abiflagsvariable can be used more easily than... if ... else ...statements:cpython/Lib/sysconfig/__init__.py
Lines 30 to 37 in 9ef9d68
If we deprecate
sys.abiflagsat all, we will have to maintain the following:( '{installed_platbase}/include/{implementation_lower}{py_version_short}' % ( installed_platbase, implementation_lower, py_version_short, ) + ('t' if 'free-threading' in sys.abi_features else '') + ('d' if 'debug' in sys.abi_features else '') + ('x' if 'some-feature-in-future' in sys.abi_features else '') )Each time we add a new build that differs in API, we will need to update this definition.
Back to the topic of the absence of
sys.abiflagson Windows. There are many places usinghasattr(sys, 'abiflags')andgetattr(sys, 'abiflags', ''). Sometimes, developers forget about this difference of presence between Windows and UNIX system. For example:An old one:
A very recent one (not fixed yet, cc @FFY00):
cpython/Tools/build/generate-build-details.py
Line 56 in 3796884
abi.flagsis included in the PEP 739 – build-details.json 1.0 — a static description file for Python build details specification. The PEP 739 states that:It should work on all platforms, including Windows.
zooba commentedon Mar 27, 2025
There should be a
sysconfigentry for this path. Nobody should be constructing it by hand, certainly not fromsysproperties (fromsysconfigproperties, potentially).sysconfig.get_config_var("ABIFLAGS")isn't going anywhere.And
sysconfigcan use whatever logic or variables it likes, because it's internal. The whole point of the module is to pull together our mess of internal settings and present a stable, easy-to-use interface for users. When users ignore it, they will find they have to work too hard, and if they ignore us when we tell them there's an easier way, there's nothing more we can do for them.This could just flat out do with better logic.
{0.name[0]}pis a ridiculous way to getcporpp(oriporjp(?) or...?) - they probably want the table frompackaging.tags.sysconfig.get_config_var("abi_thread")will give the optionaltalready, and will presumably go back to being empty when there's no longer an ABI difference. (I don't actually know where that variable came from - I didn't add it or approve it - but it's there.)What we probably need is for
sysconfig._init_non_posixto add anABIFLAGSvalue that matchesabi_thread(we don't want to touchabiflagslowercase, because that is used to calculate paths).ABIFLAGShas always been empty on Windows, AFAIK, but if it's being used in a cross-platform way already, it should be being used in the right places to generate correct filenames (though more likely not at all, sinceEXT_SUFFIXsuffices for extension modules and nowhere else would've matched a POSIX name anyway since we rarely use the same name formats).XuehaiPan commentedon Mar 27, 2025
I just want to get a hash key for the Python build which is used in the CI artifact name. It is not a formal use case for the tags. And my package only officially supports CPython and PyPy.
This is not documented.
I opened a PR for this:
ABIFLAGStosysconfig.get_config_vars()on Windows #131799sysconfig.get_config_vars()does not contain theABIFLAGSkey. You will always get the following on Windows:zooba commentedon Mar 27, 2025
Why not just use the full implementation name then? I assumed this had some user-visible aspect (e.g. what a user types to install a particular runtime version).
Yeah, again, I wasn't involved in adding that one. Whoever was clearly forgot about whether this needed to be publicly accessible. I probably would've argued for just adding
ABIFLAGS, so thanks for doing that!gh-127405: Add ABIFLAGS to sysconfig variables on Windows (GH-131799)
[-]Set `sys.abiflags` on Windows to make check free-threaded/debug builds easier[/-][+]Set `sys.abiflags` on Windows[/+]XuehaiPan commentedon Apr 11, 2025
This issue is partially resolved by #131799.
ABIFLAGStosysconfig.get_config_vars()on Windows #131799XuehaiPan commentedon Sep 8, 2025
Closing as #137476 is merged.