bpo-45405: Prevent `internal configure error` when running `configure` with recent versions of non-Apple clang. #28845

debohman · 2021-10-10T01:36:56Z

bpo-45405: Change the configure logic to function properly on darwin when the compiler outputs a platform triplet for option --print-multiarch.

https://bugs.python.org/issue45405

Change the configure logic to function properly on darwin when the compiler outputs a platform triplet for option --print-multiarch.

the-knights-who-say-ni · 2021-10-10T01:36:59Z

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@debohman

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

corona10

You may need to regenerate the configure file also.
https://devguide.python.org/setup/#regenerate-configure

bedevere-bot · 2021-10-10T07:38:07Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

ned-deily · 2021-10-10T15:42:27Z

You may need to regenerate the configure file also.
https://devguide.python.org/setup/#regenerate-configure

We discussed this on the bpo issue. I'll take care of regenerating it.

bedevere-bot · 2021-10-13T00:10:29Z

@ned-deily: Please replace # with GH- in the commit message next time. Thanks!

miss-islington · 2021-10-13T00:11:02Z

Thanks @debohman for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

miss-islington · 2021-10-13T00:11:02Z

Thanks @debohman for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

bedevere-bot · 2021-10-13T00:11:21Z

GH-28910 is a backport of this pull request to the 3.9 branch.

bedevere-bot · 2021-10-13T00:11:24Z

GH-28911 is a backport of this pull request to the 3.10 branch.

…ure`` with recent versions of non-Apple clang. (pythonGH-28845) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. Co-authored-by: Ned Deily <nad@python.org> (cherry picked from commit 9c47667) Co-authored-by: David Bohman <debohman@gmail.com>

…`configure`` with recent versions of non-Apple clang. (GH-28845) (GH-28911) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. Co-authored-by: Ned Deily <nad@python.org> (cherry picked from commit 9c47667) Co-authored-by: David Bohman <debohman@gmail.com> Automerge-Triggered-By: GH:ned-deily

…configure`` with recent versions of non-Apple clang. (GH-28845) (GH-28910) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. Co-authored-by: Ned Deily <nad@python.org> (cherry picked from commit 9c47667) Co-authored-by: David Bohman <debohman@gmail.com> Automerge-Triggered-By: GH:ned-deily

bedevere-bot · 2021-10-13T02:45:24Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 macOS 3.9 has failed when building commit 9901d15.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/108/builds/299) and take a look at the build logs.
Check if the failure is related to this commit (9901d15) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/108/builds/299

Failed tests:

test_importlib

Failed subtests:

test_multiprocessing_pool_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

406 tests OK.

10 slowest tests:

test_concurrent_futures: 4 min 54 sec
test_multiprocessing_spawn: 4 min 3 sec
test_tokenize: 3 min 28 sec
test_unparse: 3 min 21 sec
test_multiprocessing_forkserver: 2 min 33 sec
test_lib2to3: 2 min 15 sec
test_unicodedata: 2 min 3 sec
test_capi: 1 min 49 sec
test_asyncio: 1 min 41 sec
test_pickle: 1 min 1 sec

1 test failed:
test_importlib

18 tests skipped:
test_devpoll test_epoll test_gdb test_ioctl test_msilib
test_multiprocessing_fork test_ossaudiodev test_smtpnet test_spwd
test_ssl test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_importlib

Total duration: 29 min 20 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py", line 201, in main
    cache[rtype].remove(name)
KeyError: '/psm_0cd850a1'


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py", line 201, in main
    cache[rtype].remove(name)
KeyError: '/psm_041c5fae'


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 32, in __init__
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_fork.py", line 19, in __init__
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 53, in _launch
    parent_r, child_w = os.pipe()
OSError: [Errno 24] Too many open files
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 120 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-bpzw_iez': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-d0t0z8yj': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-6ge5doij': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-m1klwk16': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
---


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/test_threaded_import.py", line 258, in test_multiprocessing_pool_circular_import
    script_helper.assert_python_ok(fn)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['/Users/buildbot/buildarea/3.9.billenstein-macos/build/python.exe', '-X', 'faulthandler', '-I', '/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py']

debohman · 2021-10-13T14:45:31Z

Thank you to @ned-deily for your help on this.

…ure`` with recent versions of non-Apple clang. (python#28845) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. Co-authored-by: Ned Deily <nad@python.org>

…th recent versions of clang. (pythonGH-28845) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. The Apple Clang included with Xcode 13.3 now supports --print-multiarch causing configure to fail without this change. Co-authored-by: Ned Deily <nad@python.org> (cherry picked from commit 9c47667) Co-authored-by: David Bohman <debohman@gmail.com> Automerge-Triggered-By: GH:ned-deily (cherry picked from commit 9901d15)

…th recent versions of clang. (GH-28845) (GH-31890) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. The Apple Clang included with Xcode 13.3 now supports --print-multiarch causing configure to fail without this change. Co-authored-by: Ned Deily <nad@python.org> (cherry picked from commit 9c47667) Co-authored-by: David Bohman <debohman@gmail.com>

…th recent versions of clang. (GH-28845) (GH-31889) Change the configure logic to function properly on macOS when the compiler outputs a platform triplet for option --print-multiarch. The Apple Clang included with Xcode 13.3 now supports --print-multiarch causing configure to fail without this change. Co-authored-by: Ned Deily <nad@python.org> (cherry picked from commit 9c47667) Co-authored-by: David Bohman <debohman@gmail.com> Automerge-Triggered-By: GH:ned-deily (cherry picked from commit 9901d15) Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>

bug-45405:

894b77b

Change the configure logic to function properly on darwin when the compiler outputs a platform triplet for option --print-multiarch.

bedevere-bot added the awaiting review label Oct 10, 2021

the-knights-who-say-ni added the CLA not signed label Oct 10, 2021

corona10 requested changes Oct 10, 2021

View changes

bedevere-bot removed the awaiting review label Oct 10, 2021

bedevere-bot added the awaiting changes label Oct 10, 2021

corona10 requested a review from ned-deily Oct 10, 2021

ned-deily self-assigned this Oct 10, 2021

ned-deily added 2 commits Oct 11, 2021

Merge branch 'main' into bug-45405

e502d81

Regenerate config and add NEWS blurb

ad39591

the-knights-who-say-ni added CLA signed and removed CLA not signed labels Oct 11, 2021

Fix Sphinx markup and generalize NEWS item.

14f2ab0

ned-deily approved these changes Oct 13, 2021

View changes

bedevere-bot added awaiting merge and removed awaiting changes labels Oct 13, 2021

ned-deily changed the title ~~bpo-45405: Change the configure logic to function properly on darwin for compiler option --print-multiarch~~ bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. Oct 13, 2021

ned-deily merged commit 9c47667 into python:main Oct 13, 2021
12 checks passed

bedevere-bot removed the awaiting merge label Oct 13, 2021

ned-deily added needs backport to 3.9 needs backport to 3.10 labels Oct 13, 2021

bedevere-bot removed the needs backport to 3.9 label Oct 13, 2021

bedevere-bot removed the needs backport to 3.10 label Oct 13, 2021

python / cpython Public

bpo-45405: Prevent `internal configure error` when running `configure` with recent versions of non-Apple clang. #28845

bpo-45405: Prevent `internal configure error` when running `configure` with recent versions of non-Apple clang. #28845

debohman commented Oct 10, 2021 •

edited by bedevere-bot

the-knights-who-say-ni commented Oct 10, 2021

corona10 left a comment

bedevere-bot commented Oct 10, 2021

ned-deily commented Oct 10, 2021

bedevere-bot commented Oct 13, 2021

miss-islington commented Oct 13, 2021

miss-islington commented Oct 13, 2021

bedevere-bot commented Oct 13, 2021

bedevere-bot commented Oct 13, 2021

bedevere-bot commented Oct 13, 2021

debohman commented Oct 13, 2021

python / cpython Public

bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. #28845

bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. #28845

Conversation

debohman commented Oct 10, 2021 • edited by bedevere-bot

the-knights-who-say-ni commented Oct 10, 2021

CLA Missing

corona10 left a comment

bedevere-bot commented Oct 10, 2021

ned-deily commented Oct 10, 2021

bedevere-bot commented Oct 13, 2021

miss-islington commented Oct 13, 2021

miss-islington commented Oct 13, 2021

bedevere-bot commented Oct 13, 2021

bedevere-bot commented Oct 13, 2021

bedevere-bot commented Oct 13, 2021

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

debohman commented Oct 13, 2021

bpo-45405: Prevent `internal configure error` when running `configure` with recent versions of non-Apple clang. #28845

bpo-45405: Prevent `internal configure error` when running `configure` with recent versions of non-Apple clang. #28845

debohman commented Oct 10, 2021 •

edited by bedevere-bot