bpo-41100: Support macOS 11 and Apple Silicon #22855
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We should consider just dropping support for macOS 10.4 here to simplify the code.
This is support for ctypes on macOS/arm64 based on PR 21249 by Lawrence D'Anna (Apple). Changes: - changed __builtin_available tests from 11.0 to 10.15 - added test to setup.py for ffi_closure_alloc and use that in malloc_closure.c - Minor change in the code path for ffi_prep_closure_var (coding style change)
The preprocessor guard in the old version doesn't work, and isn't really needed (10.4 only supported 32-bit code where unsigned long is the same as uint32_t).
This also adds an option to stop building after compiling the 3th-party dependencies, as well as a script for archiving those dependencies. This makes it easier to work on the build. There are three changes to build-installer.py related to universal2 support: 1. Add 'universal2' information to build-installer.py; 2. Building OpenSSL for arm64 requires a patch at this time; 3. For some reason I had to patch the Tcl build to avoid a build error.
Needed because my previous workaround doesn't work anymore. This uses a private API, that should be made public later...
- __builtin_available is not present in the compiler, fall back to the ancient way of testing if a symbol is usable. - Logic error code in pytime.c resulted in a compile error.
…oren/cpython into macos11-deploy-earlier-branch
The tests are not entirely done yet, and something similar needs to be done for the time module.
ronaldoussoren
requested review from
abalkin,
pganssle,
tiran and
a team
as code owners
…on isn't present. I don't particularly like the code duplication, but this code should be pretty stable.
sjmulder
pushed a commit
to sjmulder/cpython
that referenced
this pull request
Nov 15, 2020
Backport of commit 4176193 to 3.9: ___ Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com> * Add support for macOS 11 and Apple Silicon (aka arm64) As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy * Support building on recent versions of macOS while deploying to older versions This allows building installers on macOS 11 while still supporting macOS 10.9.
ronaldoussoren
added a commit
to ronaldoussoren/cpython
that referenced
this pull request
Nov 15, 2020
Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com> * Add support for macOS 11 and Apple Silicon (aka arm64) As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy * Support building on recent versions of macOS while deploying to older versions This allows building installers on macOS 11 while still supporting macOS 10.9.. (cherry picked from commit 4176193) Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
|
GH-23295 is a backport of this pull request to the 3.9 branch. |
|
@maxbelanger per the tagging/metadata in the issue at https://bugs.python.org/issue41100, this will likely be ported to Python 3.8 as well. I don't know who will do it, but this is an educated guess based on how CPython's bug tracker works. Given how the 3.9 backport needed to be done manually, I imagine the 3.8 backport needs to be done as well. |
|
Porting to 3.8 is next on our list. |
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Nov 17, 2020
(Apple Silicon being their aarch64 platform.) This is backport of the same in lang/python39. Only minor changes were needed. Patches consist of: - Upstream work: python/cpython#22855 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the shared library cache there's nothing in /usr/lib. See: https://bugs.python.org/issue41116 - Addition of __arch64__ case to fix _decimal module. A very similar fix has since been committed upstream.
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Nov 17, 2020
(Apple Silicon being their aarch64 platform.) This is backport of the same in lang/python39 and lang/python38. Some parts weren't applicable in 3.7. The setup.py script needed some work on the ffi code. Otherwise, minor changes. Patches consist of: - Upstream work: python/cpython#22855 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the shared library cache there's nothing in /usr/lib. See: https://bugs.python.org/issue41116 - Addition of __arch64__ case to fix _decimal module. A very similar fix has since been committed upstream.
ronaldoussoren
added a commit
that referenced
this pull request
Nov 22, 2020
) * [3.9] bpo-41100: Support macOS 11 and Apple Silicon (GH-22855) Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com> * Add support for macOS 11 and Apple Silicon (aka arm64) As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy * Support building on recent versions of macOS while deploying to older versions This allows building installers on macOS 11 while still supporting macOS 10.9.. (cherry picked from commit 4176193) Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com> * Back port of changes to _decimal to support arm64 * temp_dir is in test.support in 3.9
|
I've started work on the backport to 3.8. That's more work that the 3.9 backport due to unrelated changes that interfere with merging. I expect to have a PR with a manual complete backport later today, and might create a manual backport of a subset of the functionality as well. |
5 tasks
mozillazg
pushed a commit
to mozillazg/pypy
that referenced
this pull request
Jan 14, 2021
On Big Sur, there is no longer a separate dylib file for each system library. Instead, they have been combined into a single shared cache file. As a result, it no longer suffices to check file existence to determine whether a system library is present. Fix this by calling _dyld_shared_cache_contains_path to check the dyld shared cache as well. The changes under lib-python/ are copied from python/cpython#22855. The _dyld_shared_cache_contains_path function is exposed to the ctypes python code using CFFI, similar to what is done in other modules, such as lib_pypy/_sha3. Fixes issue #3314.
adorilson
pushed a commit
to adorilson/cpython
that referenced
this pull request
Mar 13, 2021
Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com> * Add support for macOS 11 and Apple Silicon (aka arm64) As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy * Support building on recent versions of macOS while deploying to older versions This allows building installers on macOS 11 while still supporting macOS 10.9.
isuruf
added a commit
to isuruf/cpython
that referenced
this pull request
May 31, 2021
pythonGH-25806 was a python 3.8 backport of pythonGH-22855 to support macOS 11, but didn't backport the changes that enabled building python with a new macOS SDK and target an older macOS version. This commit adds support for that.
colesbury
pushed a commit
to colesbury/nogil
that referenced
this pull request
Oct 7, 2021
…ythonGH-23295) * [3.9] bpo-41100: Support macOS 11 and Apple Silicon (pythonGH-22855) Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com> * Add support for macOS 11 and Apple Silicon (aka arm64) As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy * Support building on recent versions of macOS while deploying to older versions This allows building installers on macOS 11 while still supporting macOS 10.9.. (cherry picked from commit 4176193) Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com> * Back port of changes to _decimal to support arm64 * temp_dir is in test.support in 3.9
ddl-jameslee
reviewed
Dec 1, 2021
| @@ -331,6 +355,12 @@ def compiler_fixup(compiler_so, cc_args): | |||
| except ValueError: | |||
| break | |||
|
|
|||
| elif not _supports_arm64_builds(): | |||
| # Look for "-arch arm64" and drop that | |||
| for idx in range(len(compiler_so)): | |||
There was a problem hiding this comment.
When idx is len(compiler_so)-1, it will throw an IndexError exception.
So, it should be for idx in range(len(compiler_so)-1):
There was a problem hiding this comment.
This PR was merged over a year ago and has long since been released to the field. Please open a new issue on the bug tracker and, if possible, provide a PR. Otherwise, it will likely be forgotten about it. Thanks!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
This merges my other two PRs, and improves on the PR that implements support for older macOS versions.
on systems that don't support his.
https://bugs.python.org/issue41100