_scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+ #70505

RobertPFischer · 2016-02-09T04:36:25Z

BPO	26317
Nosy	@terryjreedy, @ronaldoussoren, @vstinner, @ned-deily, @corona10, @websurfer5
PRs	#13306 #19681 #20176 #20177 #20182
Files	log: Log file from attempt to build Python

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = None
created_at = <Date 2016-02-09.04:36:24.611>
labels = ['OS-mac', 'build', '3.9']
title = '_scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+'
updated_at = <Date 2020-05-19.08:11:12.595>
user = 'https://bugs.python.org/RobertPFischer'

bugs.python.org fields:

activity = <Date 2020-05-19.08:11:12.595>
actor = 'Arfrever'
assignee = 'ned.deily'
closed = False
closed_date = None
closer = None
components = ['Build', 'macOS']
creation = <Date 2016-02-09.04:36:24.611>
creator = 'Robert P Fischer'
dependencies = []
files = ['41865']
hgrepos = []
issue_num = 26317
keywords = ['patch']
message_count = 27.0
messages = ['259914', '259927', '259929', '260113', '260162', '260194', '260195', '260215', '331768', '342037', '342055', '342081', '342139', '342179', '342432', '359893', '359894', '359922', '359923', '369213', '369218', '369224', '369227', '369246', '369247', '369248', '369325']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'ronaldoussoren', 'vstinner', 'ned.deily', 'Arfrever', 'python-dev', 'Robert P Fischer', 'corona10', 'Jeffrey.Kintscher']
pr_nums = ['13306', '19681', '20176', '20177', '20182']
priority = 'high'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue26317'
versions = ['Python 3.9']

RobertPFischer · 2016-02-09T04:35:33Z

Changes to OS X 10.11 render GCC's Objective-C compiler useless. However, I want to compile the main part of Python in GCC (because my C++ / Fortran Cython modules use GCC).

I tried to build Python (via MacPorts) using Clang for Objective-C and GCC for C/C++. The environment upon running ./configure included:

CC='/Users/rpfische/macports/mpgompi-4.9.3/bin/gcc-mp-4.9'
CXX='/Users/rpfische/macports/mpgompi-4.9.3/bin/gcc-mp-4.9'
OBJC='/usr/bin/clang'
OBJCXX='/usr/bin/clang++'

HOWEVER... the build still tried to use GCC to compile Objective-C, and failed miserably:

:info:destroot /Users/rpfische/macports/mpgompi-4.9.3/bin/gcc-mp-4.9 -pipe -Os -arch x86_64 -Wno-unused-result -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -o FileSettings.o -c ./FileSettings.m
:info:destroot /usr/include/objc/NSObject.h:22:4: error: unknown type name 'instancetype'
:info:destroot - (instancetype)self;
:info:destroot ^
...

Log file attached.

ronaldoussoren · 2016-02-09T11:39:38Z

The build fails while building the "Python Launcher" application. This is an Objective-C project.

If GCC doesn't support the system headers on OSX 10.11 there's nothing the Python project can do about that.

RobertPFischer · 2016-02-09T12:26:52Z

I'm trying to compile the Objective-C portions of Python (the Python
Launcher app) with clang, not GCC. Why does Python insist on using GCC,
even when I set the OBJC env variable to clang? That seems to be a problem
that the Python project CAN fix.

On Tue, Feb 9, 2016 at 6:39 AM, Ronald Oussoren <report@bugs.python.org>
wrote:

Ronald Oussoren added the comment:

The build fails while building the "Python Launcher" application. This is
an Objective-C project.

If GCC doesn't support the system headers on OSX 10.11 there's nothing the
Python project can do about that.

----------
nosy: +ronaldoussoren

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue26317\>

ronaldoussoren · 2016-02-11T16:05:06Z

The makefiles use CC throughout and don't look at and OBJC variable. Is that variable a standard way to specify an ObjC compiler in makefiles?

Make's default ruleset for compiling .m files uses $(CC) and doesn't look at $(OBJC).

I wouldn't be opposed to a patch that makes it possible to specify an ObjC compiler, could you write such a patch? BTW. The patch should also update the autoconf script (use AC_PROG_OBJC).

RobertPFischer · 2016-02-12T04:45:52Z

The makefiles use CC throughout and don't look at and OBJC variable. Is
that variable a standard way to specify an ObjC compiler in makefiles?

I believe that OBJC and OBJCFLAGS are standard for Autoconf/Automake:

https://www.gnu.org/software/automake/manual/html_node/Objective-C-Support.html

https://www.gnu.org/software/autoconf/manual/autoconf-2.61/html_node/Output-Variable-Index.html#Output-Variable-Index

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=using%20objective%20c%20with%20autoconf%20automake

I wouldn't be opposed to a patch that makes it possible to specify an ObjC

compiler, could you write such a patch? BTW. The patch should also update
the autoconf script (use AC_PROG_OBJC).

Sorry, I don't do Autotools --- a software package that came to us directly
from the seventh level of Hell. CMake's main claim to fame is it's not as
bad.

terryjreedy · 2016-02-12T19:34:30Z

3.4 only gets security fixes

RobertPFischer · 2016-02-12T19:40:22Z

Could this be fixed on 3.5 and 3.6?

On Fri, Feb 12, 2016 at 2:34 PM, Terry J. Reedy <report@bugs.python.org>
wrote:

Terry J. Reedy added the comment:

3.4 only gets security fixes

----------
nosy: +terry.reedy
stage: -> needs patch
type: -> compile error
versions: +Python 3.5, Python 3.6 -Python 3.4

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue26317\>

terryjreedy · 2016-02-12T23:47:17Z

When you reply by email to an email, please snip off the quote, except for possible a line that you are responding to. The disposition of this issue for 3.5+ is up to Ned or Ronald.

terryjreedy · 2018-12-13T19:58:50Z

3.5 is and 3.6 will soon be on security fix only status. Robert, please verify that this is still an issue with 3.7 and the current build files.

websurfer5 · 2019-05-10T02:00:17Z

This is still an issue with the 3.7 and the master branches. The Objective-C portion of the build still fails with gcc 8.3.0 on OS X Mojave (14.x). Specifying OBJC and OBJC++ on the configure command line does nothing because there is nothing in the script using the options.

websurfer5 · 2019-05-10T05:41:40Z

Module/_scproxy.c compiles cleanly with clang 10.0.0 and fails with GNU gcc 8.3.0. Below is the relevant compiler error:

building '_scproxy' extension
gcc-8 -Wno-unused-result -Wsign-compare -g -Og -Wall -pipe -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I../cpython/Include/internal -I../cpython/Include -IObjects -IPython -I. -I/usr/local/opt/zlib/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/ossp-uuid/1.6.2_2/include -I/usr/local/include -I/Users/jeff/sandbox/src/python3.7/cpython/Include -I/Users/jeff/sandbox/src/python3.7/build -c /Users/jeff/sandbox/src/python3.7/cpython/Modules/_scproxy.c -o build/temp.macosx-10.14-x86_64-3.8-pydebug/Users/jeff/sandbox/src/python3.7/cpython/Modules/_scproxy.o
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/AuthSession.h:32,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/Security.h:42,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SCPreferences.h:35,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SystemConfiguration.h:126,
                 from /Users/jeff/sandbox/src/python3.7/cpython/Modules/_scproxy.c:6:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/Authorization.h:193:7:error: variably modified 'bytes' at file scope
  char bytes[kAuthorizationExternalFormLength];
       ^~~~~
building 'zlib' extension

ned-deily · 2019-05-10T17:23:13Z

THe _scproxy.c compile error is a separate issue. The problem arises in an Apple-supplied include file and there are many reports on the web of clang vs gcc differences like this. I'm not sure what we could or should do about it other than forcing _scproxy to always be compiled with clang or figure out a way to avoid use of that include file hierarchy.

websurfer5 · 2019-05-10T21:53:03Z

My mistake was not adding

--enable-framework=${HOME}/sandbox/src/python3.7/inst

to the configure script and then running

make
make install

(I know... read the README file)

gcc 8.3.0 doesn't like the Xcode system headers included by the .m files. I will dig into this now that I am using the proper build steps.

As a side note, I find it counter-intuitive that Mac/Makefile only gets invoked by 'make install' to build the Objective-C source files. IMHO they should be built by "make" and then installed by "make install". Forcing a build and install in one step can be dangerous and accidentally leave the system in an undesirable state when the build process fails with a partial install. Also, --enable-framework should use a subpath of --prefix by default for the framework installation instead of defaulting to /Library/Frameworks regardless of the --prefix setting.

websurfer5 · 2019-05-11T10:18:48Z

I got it to build PythonLauncher using clang and everything else using gcc 8.3.0 by making a few configure script and makefile changes. Unfortunately, "make install" still fails because a python script that requires the unbuilt _scproxy module crashes. I'll work on fixing that next.

websurfer5 · 2019-05-14T05:26:57Z

I submitted a pull request that fixes both the PythonLauncher build issue and the _scproxy module build issue. They are related because _scproxy is required to install PythonLauncher.

corona10 · 2020-01-13T10:00:37Z

Any progress?

corona10 · 2020-01-13T10:03:07Z

I met this issue during I compile with gcc9 on my local mac machine.
This issue should be fixed..

terryjreedy · 2020-01-13T18:16:58Z

The progress was the PR. I just fixed the merge conflicts, I believe correctly. Please test and review.

ned-deily · 2020-01-13T18:19:59Z

Thanks, Terry. I'll look at it shortly.

ned-deily · 2020-05-18T13:17:25Z

New changeset 0da5466 by Ned Deily in branch 'master':
bpo-26317: Support OBJC and OBJCXX configure command line variables (GH-20176)
0da5466

ned-deily · 2020-05-18T13:38:16Z

Thanks everyone for their comments and reviews and a very big thank you to Jeffrey for working through the details and providing a solid PR. The change will appear in the 3.9.0b1 pre-release.

vstinner · 2020-05-18T14:06:45Z

This change broke the Python compilation on FreeBSD:
https://buildbot.python.org/all/#/builders/152/builds/855

Either revert it (and re-apply a fixed change after beta1), or fix it. I suggest to block Python 3.9 beta1 release until it's resolved. I consider that FreeBSD is a supported platform.

ned-deily · 2020-05-18T14:08:35Z

Sorry about that! Looking now.

ned-deily · 2020-05-18T15:31:37Z

New changeset 951ab58 by Ned Deily in branch 'master':
Revert "bpo-26317: Support OBJC and OBJCXX configure command line variables (GH-20176)" (GH-20182)
951ab58

ned-deily · 2020-05-18T15:32:22Z

I'm not sure yet exactly what the problem is but it's almost certainly a difference between GNU make and BSD make and something that needs to be checked on FreeBSD. With the 3.9.0b1 cutoff coming very soon, I likely won't be able to test on FreeBSD before then so I have reverted the change for now. Sigh!

vstinner · 2020-05-18T15:32:49Z

I tested manually and I confirm that FreeBSD builds again, thanks :-)

Arfrever · 2020-05-19T08:11:13Z

Also there was FLAFS typo in this line, which is probably not needed at all:
OBJCFLAFS=@OBJCFLAGS@

AC_SUBST(OBJCXX) was added, but there was no usage of @OBJCXX@ anywhere...

erlend-aasland · 2022-05-19T22:23:51Z

Is this still a valid issue? @corona10?

ned-deily · 2022-05-19T22:34:00Z

The original PR to add separate OBJC config variables was reverted due to the buildbot failures and was never reapplied. So I expect it is still an issue for those who want to try to build Python on macOS with gcc. It's not something we recommend and is perhaps of even less importance with the introduction of Apple Silicon Macs and their build dependencies on the Apple compiler tool chain. I don't have any immediate plans to pursue it so I'm deassigning myself if someone else wants to jump in.

ned-deily · 2022-05-19T22:44:42Z

Also, the PR would need to be updated to deal with the subsequent changes to Modules/Setup et al to allow building of _scproxy without Distutils.

erlend-aasland · 2022-05-19T23:06:25Z

I'm marking it as pending (close). If Dong-hee or anyone else want to pursue this, feel free to remove the label and pick up the loose ends.

erlend-aasland · 2022-07-20T20:19:11Z

No-one has jumped in here. Closing this; feel free to reopen if you disagree.

RobertPFischer mannequin added the build label Feb 9, 2016

ronaldoussoren added the OS-mac label Feb 11, 2016

terryjreedy added the build label Feb 12, 2016

terryjreedy added the 3.7 label Dec 13, 2018

websurfer5 mannequin added the 3.8 label May 10, 2019

ned-deily self-assigned this Jan 13, 2020

ned-deily added 3.9 and removed 3.7 3.8 labels May 18, 2020

ned-deily closed this as completed May 18, 2020

ned-deily changed the title ~~Build Problem with GCC + Macintosh OS X 10.11 El Capitain~~ _scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+ May 18, 2020

vstinner added the release-blocker label May 18, 2020

vstinner reopened this May 18, 2020

ned-deily removed the release-blocker label May 18, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

erlend-aasland removed the 3.9 label May 19, 2022

ned-deily removed their assignment May 19, 2022

erlend-aasland added the pending label May 19, 2022

erlend-aasland closed this as not planned Jul 20, 2022

erlend-aasland removed the pending label Jul 20, 2022

_scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+ #70505

_scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+ #70505

RobertPFischer mannequin commented Feb 9, 2016

RobertPFischer mannequin commented Feb 9, 2016

ronaldoussoren commented Feb 9, 2016

RobertPFischer mannequin commented Feb 9, 2016

ronaldoussoren commented Feb 11, 2016

RobertPFischer mannequin commented Feb 12, 2016

terryjreedy commented Feb 12, 2016

RobertPFischer mannequin commented Feb 12, 2016

terryjreedy commented Feb 12, 2016

terryjreedy commented Dec 13, 2018

websurfer5 mannequin commented May 10, 2019

websurfer5 mannequin commented May 10, 2019

ned-deily commented May 10, 2019

websurfer5 mannequin commented May 10, 2019

websurfer5 mannequin commented May 11, 2019

websurfer5 mannequin commented May 14, 2019

corona10 commented Jan 13, 2020

corona10 commented Jan 13, 2020

terryjreedy commented Jan 13, 2020

ned-deily commented Jan 13, 2020

ned-deily commented May 18, 2020

ned-deily commented May 18, 2020

vstinner commented May 18, 2020

ned-deily commented May 18, 2020

ned-deily commented May 18, 2020

ned-deily commented May 18, 2020

vstinner commented May 18, 2020

Arfrever mannequin commented May 19, 2020

erlend-aasland commented May 19, 2022

ned-deily commented May 19, 2022 •

edited

ned-deily commented May 19, 2022

erlend-aasland commented May 19, 2022

erlend-aasland commented Jul 20, 2022

_scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+ #70505

_scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+ #70505

Comments

RobertPFischer mannequin commented Feb 9, 2016

RobertPFischer mannequin commented Feb 9, 2016

ronaldoussoren commented Feb 9, 2016

RobertPFischer mannequin commented Feb 9, 2016

ronaldoussoren commented Feb 11, 2016

RobertPFischer mannequin commented Feb 12, 2016

terryjreedy commented Feb 12, 2016

RobertPFischer mannequin commented Feb 12, 2016

terryjreedy commented Feb 12, 2016

terryjreedy commented Dec 13, 2018

websurfer5 mannequin commented May 10, 2019

websurfer5 mannequin commented May 10, 2019

ned-deily commented May 10, 2019

websurfer5 mannequin commented May 10, 2019

websurfer5 mannequin commented May 11, 2019

websurfer5 mannequin commented May 14, 2019

corona10 commented Jan 13, 2020

corona10 commented Jan 13, 2020

terryjreedy commented Jan 13, 2020

ned-deily commented Jan 13, 2020

ned-deily commented May 18, 2020

ned-deily commented May 18, 2020

vstinner commented May 18, 2020

ned-deily commented May 18, 2020

ned-deily commented May 18, 2020

ned-deily commented May 18, 2020

vstinner commented May 18, 2020

Arfrever mannequin commented May 19, 2020

erlend-aasland commented May 19, 2022

ned-deily commented May 19, 2022 • edited

ned-deily commented May 19, 2022

erlend-aasland commented May 19, 2022

erlend-aasland commented Jul 20, 2022

ned-deily commented May 19, 2022 •

edited