Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS linker warnings in macOS ventura #97524

Open
pablogsal opened this issue Sep 24, 2022 · 4 comments
Open

macOS linker warnings in macOS ventura #97524

pablogsal opened this issue Sep 24, 2022 · 4 comments

Comments

@pablogsal
Copy link
Member

pablogsal commented Sep 24, 2022

Turns out ld shipped with Xcode 14+ emits a warning when using -undefined dynamic_lookup.`

ld: warning: -undefined dynamic_lookup may not work with chained fixups

Some investigation reveals that in fact -undefined dynamic_lookup doesn't work when:

  • Link a shared library with the option
  • Link it with a program that uses the chained-fixup introduced from macOS 12 and iOS 15

This is because -undefined dynamic_lookup uses lazy-bindings and they won't be bound while dyld fixes-up by traversing chained-fixup info.

However, we build extension modules with as bundles (-bundle) and they are loaded only through
dlopen, so it's safe to use -undefined dynamic_lookup in theory. So the warning produced by ld64 is
likely to be a false-positive.

ld64 also provides the -bundle_loader <executable> option, which allows resolving symbols defined in the executable symbol table while linking. It behaves almost the same with -undefined dynamic_lookup, but it makes the following changes:

  • Require that unresolved symbols among input objects must be defined in the executable.
  • Lazy symbol binding will lookup only the symbol table of the bundle loader executable. (-undefined dynamic_lookup lookups all symbol tables as flat namespace)

See "New Features" subsection under "Linking" section for chained fixup
developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes for more information.

Also, check the same problem in ruby where most of this information is taken from.

@pablogsal
Copy link
Member Author

pablogsal commented Sep 24, 2022

CC @ambv @ned-deily

@pablogsal
Copy link
Member Author

pablogsal commented Sep 24, 2022

Unfortunately this warning also propagates to all extension modules built with the given Python.

@ned-deily
Copy link
Member

ned-deily commented Sep 24, 2022

@ronaldoussoren

@ronaldoussoren
Copy link
Contributor

ronaldoussoren commented Sep 25, 2022

-bundle_loader doesn't work with shared library builds (including the framework build), as the option does exactly as described in the documentation: It looks for symbols in the loader executable, not in shared library dependencies of that binary. I found this when looking into -bundle_loader to get link time errors when using non-existing symbols instead of only getting those when using an extension.

Linking with libpython is also problematic because on macOS shared libraries are referenced by absolute path which means linking with libpython ties the extension to a specific interpreter (e.g. makes it impossible to build an extension using the Python.org installer and load it using a homebrew install). For the user this will result in seemingly random crashes (due to libpython being loaded twice while only one is initialised).

I haven't spend time on looking into this particular issue yet, with some luck there's another option beyond disabling chained lookups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants