Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ctypes][linux] ctypes.util.find_library() crash instead of return None #114257

Closed
Jacobfaib opened this issue Jan 18, 2024 · 3 comments
Closed

[ctypes][linux] ctypes.util.find_library() crash instead of return None #114257

Jacobfaib opened this issue Jan 18, 2024 · 3 comments
Labels
3.11 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes topic-ctypes type-bug An unexpected behavior, bug, or error

Comments

@Jacobfaib
Copy link

Jacobfaib commented Jan 18, 2024 •
edited by bedevere-app bot

Bug report

Bug description:

The problem:

ctypes.util.find_library() raises an exception on certain inputs when it cannot find it.

Expected behavior:

No exception, return None as advertised if it cannot find the library.

Reproducer:

# on python 3.10, but same deal on other versions too
>>> import ctypes.util
>>> ctypes.util.find_library('libgomp')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/lib/python3.10/ctypes/util.py", line 351, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
  File "/path/to/lib/python3.10/ctypes/util.py", line 148, in _findLib_gcc
    if not _is_elf(file):
  File "/path/to/lib/python3.10/ctypes/util.py", line 100, in _is_elf
    with open(filename, 'br') as thefile:
FileNotFoundError: [Errno 2] No such file or directory: b'liblibgomp.a'

Note that the crash does not occur if you put nonsense in:

  1. find_library('asdadsas') -> OK
  2. find_library('libasdasdasd') -> OK

The problem appears to be a bit more contrived. Python is attempting to parse the output from gcc, which in this case is:

/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/Scrt1.o
/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/12/crtbeginS.o
/usr/bin/ld: cannot find -llibgomp: No such file or directory
/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/12/libgomp.a use -l:libgomp.a or rename it to liblibgomp.a
/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/12/libgcc_s.so
/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/libgcc_s.so.1
/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a
collect2: error: ld returned 1 exit status

But then the parser is getting confused trying to parse the useful diagnostic message mentioning libgomp.a.

Machine details (from platform.uname())

uname_result(system='Linux', node='iblis', release='6.5.0-14-generic', version='#14~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 20 18:15:30 UTC 2', machine='x86_64')

CPython versions tested on:

3.10, 3.11, 3.12

Operating systems tested on:

Linux

Linked PRs
@Jacobfaib Jacobfaib added the type-bug An unexpected behavior, bug, or error label Jan 18, 2024
@Jacobfaib
Copy link
Author

cc @manopapad

@manopapad
Copy link

manopapad commented Jan 18, 2024 •
edited

Full output with some prints (python pulled from conda-forge):

(test) iblis:~/jacob/legate.core> python --version
Python 3.12.1
(test) iblis:~/jacob/legate.core> cat a.py
import ctypes.util
ctypes.util.find_library('libgomp')

(test) iblis:~/jacob/legate.core> python a.py
_findLib_gcc(name=libgomp)
expr=b'[^\\(\\)\\s]*liblibgomp\\.[^\\(\\)\\s]*'
trace=b'/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/Scrt1.o\n/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/crti.o\n/usr/lib/gcc/x86_64-linux-gnu/12/crtbeginS.o\n/usr/bin/ld: cannot find -llibgomp: No such file or directory\n/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/12/libgomp.a use -l:libgomp.a or rename it to liblibgomp.a\n/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a\n/usr/lib/gcc/x86_64-linux-gnu/12/libgcc_s.so\n/usr/lib/gcc/x86_64-linux-gnu/12/../../../x86_64-linux-gnu/libgcc_s.so.1\n/usr/lib/gcc/x86_64-linux-gnu/12/libgcc.a\ncollect2: error: ld returned 1 exit status\n'
res=[b'liblibgomp.a']
Traceback (most recent call last):
  File "/home/mpapadakis/jacob/legate.core/a.py", line 2, in <module>
    ctypes.util.find_library('libgomp')
  File "/home/mpapadakis/mambaforge/envs/test/lib/python3.12/ctypes/util.py", line 355, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
                ^^^^^^^^^^^^^^^^^^
  File "/home/mpapadakis/mambaforge/envs/test/lib/python3.12/ctypes/util.py", line 152, in _findLib_gcc
    if not _is_elf(file):
           ^^^^^^^^^^^^^
  File "/home/mpapadakis/mambaforge/envs/test/lib/python3.12/ctypes/util.py", line 100, in _is_elf
    with open(filename, 'br') as thefile:
         ^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: b'liblibgomp.a'

I assume the double "lib" is intentional, and this call was meant to fail, and just ended up mis-parsing an unrelated part of the output diagnostic message.

@aisk
Copy link
Member

aisk commented Jan 21, 2024

The find_library function will invoke gcc and search for any matches to the pattern lib{name} (in this case, liblibgomp) in its output.

In the new version of gcc, it will display a recommendation tip in case you add the unnecessary lib prefix, like:

/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/12/libgomp.a use -l:libgomp.a or rename it to liblibgomp.a

If a matching result, such as liblibgomp.a, is found, it will be used as an argument to _is_elf. Unfortunately, if the file does not exist or for any other reason when it's opened, it will raise an error.

I'm not sure if any other OSError should be ignored, but at least FileNotFoundError should be ignored in this case.

aisk added a commit to aisk/cpython that referenced this issue Jan 21, 2024
@aisk
pythongh-114257: Ignore the FileNotFound error in `ctypes.util._is_…
31701e1 
…elf`
@bedevere-app bedevere-app bot mentioned this issue Jan 21, 2024
Merged
@serhiy-storchaka serhiy-storchaka added 3.11 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes labels Jan 22, 2024
serhiy-storchaka pushed a commit that referenced this issue Jan 22, 2024
@aisk
gh-114257: Ignore the FileNotFound error in ctypes.util._is_elf() (GH…
7fc51c3 
…-114394)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jan 22, 2024
@aisk @miss-islington
pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…
d256c8c 
…() (pythonGH-114394)

(cherry picked from commit 7fc51c3)

Co-authored-by: AN Long <aisk@users.noreply.github.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jan 22, 2024
@aisk @miss-islington
pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…
736c0cb 
…() (pythonGH-114394)

(cherry picked from commit 7fc51c3)

Co-authored-by: AN Long <aisk@users.noreply.github.com>
This was referenced Jan 22, 2024
Merged
Merged
serhiy-storchaka pushed a commit that referenced this issue Jan 22, 2024
@miss-islington @aisk
[3.12] gh-114257: Ignore the FileNotFound error in ctypes.util._is_el…
ed567c1 
…f() (GH-114394) (GH-114444)

(cherry picked from commit 7fc51c3)

Co-authored-by: AN Long <aisk@users.noreply.github.com>
serhiy-storchaka pushed a commit that referenced this issue Jan 22, 2024
@miss-islington @aisk
[3.11] gh-114257: Ignore the FileNotFound error in ctypes.util._is_el…
f5d9980 
…f() (GH-114394) (GH-114445)

(cherry picked from commit 7fc51c3)

Co-authored-by: AN Long <aisk@users.noreply.github.com>
aisk added a commit to aisk/cpython that referenced this issue Feb 11, 2024
@aisk
pythongh-114257: Ignore the FileNotFound error in ctypes.util._is_elf…
79c7dec 
…() (pythonGH-114394)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.11 only security fixes 3.12 bugs and security fixes 3.13 new features, bugs and security fixes topic-ctypes type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@manopapad @aisk @serhiy-storchaka @Jacobfaib @Eclips4