Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Python with C11 #91731

Open
vstinner opened this issue Apr 20, 2022 · 28 comments
Open

Build Python with C11 #91731

vstinner opened this issue Apr 20, 2022 · 28 comments
Assignees
Labels
release-blocker type-bug type-feature

Comments

@vstinner
Copy link
Member

@vstinner vstinner commented Apr 20, 2022

PEP 7 was recently updated to require a C11 compiler (without optional features) to build Python. I propose to now implement this new requirement:

  • Build Python with -std=c11 rather than -std=c99
  • Use static_assert() of <assert.h>

It's already documented in What's New in Python 3.11:

Building Python now requires a C11 compiler without optional C11 features. (Contributed by Victor Stinner in bpo-46656.

https://docs.python.org/dev/whatsnew/3.11.html#build-changes

@vstinner vstinner added type-bug type-feature labels Apr 20, 2022
vstinner added a commit to vstinner/cpython that referenced this issue Apr 20, 2022
Python is now built with "-std=c11" compiler option, rather than
"-std=c99".
@corona10
Copy link
Member

@corona10 corona10 commented Apr 20, 2022

+1

vstinner added a commit to vstinner/cpython that referenced this issue Apr 20, 2022
Python is now built with "-std=c11" compiler option, rather than
"-std=c99".
vstinner added a commit that referenced this issue Apr 20, 2022
Python is now built with "-std=c11" compiler option, rather than
"-std=c99".
vstinner added a commit to vstinner/cpython that referenced this issue Apr 20, 2022
Python 3.11 now uses C11 standard which adds static_assert()
to <assert.h>.

* In pytime.c, replace Py_BUILD_ASSERT() with preprocessor checks on
  SIZEOF_TIME_T with #error.
* On macOS, py_mach_timebase_info() now accepts timebase members with
  the same size than _PyTime_t.
* py_get_monotonic_clock() now saturates GetTickCount64() to
  _PyTime_MAX: GetTickCount64() is unsigned, whereas _PyTime_t is
  signed.
vstinner added a commit that referenced this issue Apr 20, 2022
Python 3.11 now uses C11 standard which adds static_assert()
to <assert.h>.

* In pytime.c, replace Py_BUILD_ASSERT() with preprocessor checks on
  SIZEOF_TIME_T with #error.
* On macOS, py_mach_timebase_info() now accepts timebase members with
  the same size than _PyTime_t.
* py_get_monotonic_clock() now saturates GetTickCount64() to
  _PyTime_MAX: GetTickCount64() is unsigned, whereas _PyTime_t is
  signed.
@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 20, 2022

I merged 2 PRs:

I close the issue.

@gvanrossum
Copy link
Member

@gvanrossum gvanrossum commented Apr 20, 2022

What about Windows? @zooba are we already building with C11 there?

@zooba
Copy link
Member

@zooba zooba commented Apr 20, 2022

What about Windows? @zooba are we already building with C11 there?

Apparently, or CI would have broken. I guess we'll find out when people start using new language features.

@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 20, 2022

What about Windows? @zooba are we already building with C11 there?

When the PEP 7 change was discussed on python-dev, it was said that MSVC supports C11 without optional features.

My PR got merged and the CI is fine. Do you have remaining concerns about MSVC?

The blog post mentions the /std:c11 option: https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=msvc-170 It doesn't seem to be used by Python currently.

I only these /std options:

PCbuild/python_uwp.vcxproj:93:      <AdditionalOptions>/EHsc /std:c++17 %(AdditionalOptions)</AdditionalOptions>
PCbuild/pythonw_uwp.vcxproj:93:      <AdditionalOptions>/EHsc /std:c++17 %(AdditionalOptions)</AdditionalOptions>

The blog post also mentions the /Zc:preprocessor option to switch the preprocessor to C11 mode ("conformant preprocessor"): https://docs.microsoft.com/en-us/cpp/build/reference/zc-preprocessor?view=msvc-170 This option doesn't seem to be currently used by Python.

@gvanrossum
Copy link
Member

@gvanrossum gvanrossum commented Apr 20, 2022

My remaining concern is that I tried to use static_assert() in #32387 and it broke on Windows. I've merged the latest main and it still breaks. I'll just assume that that is a different problem so I'll move on. (If you want to look into it, details are in that issue.)

@zooba
Copy link
Member

@zooba zooba commented Apr 20, 2022

The blog post mentions the /std:c11 option: https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=msvc-170 It doesn't seem to be used by Python currently.

Yeah, this probably needs to be added (in pyproject.props). I don't think it will matter if it's included twice in the C++ projects, but that would be the biggest issue.

@gvanrossum
Copy link
Member

@gvanrossum gvanrossum commented Apr 21, 2022

I reopen the issue.

@gvanrossum gvanrossum reopened this Apr 21, 2022
@sweeneyde
Copy link
Member

@sweeneyde sweeneyde commented Apr 21, 2022

It looks like AMD64 FreeBSD Non-Debug 3.x is now failing like this:

-- Objects/longobject.o ---
cc -pthread -c  -DNDEBUG -g -fwrapv -O3 -Wall -UNDEBUG -O2 -pipe  -std=c11 -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include    -DPy_BUILD_CORE -o Objects/longobject.o Objects/longobject.c
Objects/longobject.c:776:5: error: implicit declaration of function 'static_assert' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    static_assert(PyLong_SHIFT <= sizeof(unsigned long) * 8,
    ^

@arhadthedev
Copy link
Contributor

@arhadthedev arhadthedev commented Apr 21, 2022

@sweeneyde

'static_assert' is invalid in C99

Is config regenerated in a commit pulled by the buildbot?

@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 21, 2022

It looks like AMD64 FreeBSD Non-Debug 3.x is now failing like this:

Right, I just noticed that: I created #91782 to track this issue.

Sadly, this specific FreeBSD buildbot worker failed on the Git step but the CI job was marked as successful on my PR :-(

@kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented Apr 21, 2022

_Py_NO_RETURN macro can also be changed to use noreturn from stdnoreturn.h.
See https://www.gnu.org/software/gnulib/manual/html_node/stdnoreturn_002eh.html

@vstinner
Copy link
Member Author

@vstinner vstinner commented Apr 21, 2022

_Py_NO_RETURN macro can also be changed to use noreturn from stdnoreturn.h.

If you consider that it's worth it, please open a separated issue.

I already got enough issues with static_assert(), so I prefer to restrict this issue to that :-)

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

Seems that Python 3.11.0 beta 1 doesn't build with gcc 8 in RHEL 6:


/tmp/python3.11-3.11.0~beta1-0/Objects/longobject.c: In function 'bit_length_digit':
 /tmp/python3.11-3.11.0~beta1-0/Objects/longobject.c:776:5: error: implicit declaration of function 'static_assert' [-Werror=implicit-function-declaration]
  776 |     static_assert(PyLong_SHIFT <= sizeof(unsigned long) * 8,
          |     ^~~~~~~~~~~~~

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

I'm marking this as release blocker as there may be multiple systems affected

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

Seems that the C11 symbol is _Static_assert not static_assert. The later is a C++ symbol:

https://gcc.gnu.org/wiki/C11Status

static_assert requires glibc 2.16 or newer.

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

For instance, in RHEL 6 /usr/include/assert.h doesn't provide static_assert. This means that manylinux2010 won't be able to compile Python 3.11.

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

Opened #92559

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

@mdickinson
Copy link
Member

@mdickinson mdickinson commented May 9, 2022

Seems that the C11 symbol is _Static_assert not static_assert.

static_assert should be valid in standard C11, too. It's a macro in assert.h. See C11 §7.2p3, which says:

The macro static_assert expands to _Static_assert.

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 9, 2022

static_assert should be valid in standard C11, too. It's a macro in assert.h. See C11 §7.2p3, which says:

That's only on glibc 2.16 or newer. GCC itself only provides _Static_assert

@mdickinson
Copy link
Member

@mdickinson mdickinson commented May 9, 2022

Agreed that the C11 requirement apparently mandates glibc >= 2.16. But static_assert is a C11 feature, not just a C++ feature.

@vstinner
Copy link
Member Author

@vstinner vstinner commented May 9, 2022

static_assert should be valid in standard C11, too. It's a macro in assert.h. See C11 §7.2p3, which says:

There is the theory, and the practice. Did you see the FreeBSD issue about the practice? :-) #91782

miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 9, 2022
…cs (pythonGH-92559)

(cherry picked from commit f0614ca)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
pablogsal added a commit that referenced this issue May 9, 2022
…-92559) (#92566)

(cherry picked from commit f0614ca)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
@jmroot
Copy link
Contributor

@jmroot jmroot commented May 10, 2022

macOS 10.10 and older also don't have static_assert defined even when using a C11 compiler.

@vstinner
Copy link
Member Author

@vstinner vstinner commented May 10, 2022

macOS 10.10 and older also don't have static_assert defined even when using a C11 compiler.

What is your C compiler? What is your error message?

Can you try to add this code at the top of pymacro.h? Does it fix your build issue?

#ifndef static_assert
#  define static_assert _Static_assert
#endif

pablogsal added a commit that referenced this issue May 10, 2022
* Fix typo in pymacro.h

* Update Include/pymacro.h

Co-authored-by: Victor Stinner <vstinner@python.org>

Co-authored-by: Victor Stinner <vstinner@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 10, 2022
* Fix typo in pymacro.h

* Update Include/pymacro.h

Co-authored-by: Victor Stinner <vstinner@python.org>

Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 4e6da50)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
miss-islington added a commit that referenced this issue May 10, 2022
* Fix typo in pymacro.h

* Update Include/pymacro.h

Co-authored-by: Victor Stinner <vstinner@python.org>

Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 4e6da50)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
@jmroot
Copy link
Contributor

@jmroot jmroot commented May 10, 2022

What is your C compiler? What is your error message?

Apple clang-700.1.81 and clang 11 from llvm.org both produce the same error:

Objects/longobject.c:776:5: error: implicit declaration of function 'static_assert' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    static_assert(PyLong_SHIFT <= sizeof(unsigned long) * 8,
    ^

Can you try to add this code at the top of pymacro.h? Does it fix your build issue?

Yes, that does fix the build. (Technically I just deleted defined(__FreeBSD__) && from the existing check.)

@pablogsal
Copy link
Member

@pablogsal pablogsal commented May 10, 2022

We probably need to add another case. I will prepare a PR. Alternatively, given that this is a macro, we can check if is defined and redefine it ourselves.

@vstinner
Copy link
Member Author

@vstinner vstinner commented May 10, 2022

Oh, clang 11 didn't support calling static_assert() in the function scope. It was fixed at 2021-01-29 (only in clang 12): https://bugs.llvm.org/show_bug.cgi?id=48904

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-blocker type-bug type-feature
Projects
Status: Todo
Development

No branches or pull requests

10 participants