Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hashlib.algorithms_available lists algorithms that are not available in OpenSSL 3.0 default provider #91257

Open
tiran opened this issue Mar 23, 2022 · 7 comments
Open

hashlib.algorithms_available lists algorithms that are not available in OpenSSL 3.0 default provider #91257

tiran opened this issue Mar 23, 2022 · 7 comments
Labels
3.9 3.10 3.11 expert-SSL extension-modules stdlib type-bug

Comments

@tiran
Copy link
Member

@tiran tiran commented Mar 23, 2022

BPO 47101
Nosy @tiran, @miss-islington
PRs
  • #32076
  • #32084
  • #32085
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2022-03-23.14:28:49.732>
labels = ['type-bug', '3.9', '3.10', '3.11', 'extension-modules', 'library']
title = 'hashlib.algorithms_available lists algorithms that are not available in OpenSSL 3.0 default provider'
updated_at = <Date 2022-03-23.21:15:34.062>
user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2022-03-23.21:15:34.062>
actor = 'christian.heimes'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules', 'Library (Lib)']
creation = <Date 2022-03-23.14:28:49.732>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 47101
keywords = ['patch']
message_count = 6.0
messages = ['415877', '415878', '415880', '415907', '415909', '415912']
nosy_count = 2.0
nosy_names = ['christian.heimes', 'miss-islington']
pr_nums = ['32076', '32084', '32085']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue47101'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Mar 23, 2022

    Hubert Kario wrote in https://bugzilla.redhat.com/show_bug.cgi?id=2054702

    Description of problem:
    The hashlib.algorithms_available set includes algorithms like ripemd160 and whirlpool, those algorithms are not usable unless openssl legacy provider is loaded. Since it's not loaded, and the hashlib module won't load it, any attempt to use them fails.

    Version-Release number of selected component (if applicable):
    python3-3.9.10-1.el9.x86_64
    openssl-3.0.1-5.el9.x86_64

    How reproducible:
    always

    Steps to Reproduce:
    0. start python3

    1. from hashlib import algorithms_available
    2. algorithms_available
    3. import hashlib
    4. a = {(name, hashlib.new(name).digest_size) for name in algorithms_available}

    Actual results:
    {'sha3_384', 'blake2s', 'sha384', 'sha512_224', 'md5', 'sha3_512', 'md5-sha1', 'sha3_256', 'shake_128', 'sm3', 'sha256', 'sha512', 'sha1', 'shake_256', 'blake2b', 'whirlpool', 'sha512_256', 'sha3_224', 'sha224', 'ripemd160', 'md4'}

    Traceback (most recent call last):
  File "/usr/lib64/python3.9/hashlib.py", line 164, in __hash_new
    return _hashlib.new(name, data, **kwargs)
ValueError: [digital envelope routines] unsupported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <setcomp>
  File "/usr/lib64/python3.9/hashlib.py", line 170, in __hash_new
    return __get_builtin_constructor(name)(data)
  File "/usr/lib64/python3.9/hashlib.py", line 127, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type whirlpool

    Expected results:
    {'sha3_384', 'blake2s', 'sha384', 'sha512_224', 'md5', 'sha3_512', 'md5-sha1', 'sha3_256', 'shake_128', 'sm3', 'sha256', 'sha512', 'sha1', 'shake_256', 'blake2b', 'sha512_256', 'sha3_224', 'sha224'}

    {('blake2b', 64), ('sha512', 64), ('md5-sha1', 36), ('sha3_512', 64), ('md5', 16), ('sha224', 28), ('shake_128', 0), ('sm3', 32), ('blake2s', 32), ('sha1', 20), ('shake_256', 0), ('sha512_256', 32), ('sha3_224', 28), ('sha3_256', 32), ('sha3_384', 48), ('sha384', 48), ('sha256', 32), ('sha512_224', 28)}

    Additional info:
    If the legacy provider is loaded, then the algorithms should be listed and should work.

    It may be caused by Python using the deprecated EVP_MD_do_all() method instead of the EVP_MD_do_all_provided() method

    @tiran tiran added 3.9 3.10 3.11 extension-modules stdlib type-bug labels Mar 23, 2022
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Mar 23, 2022

    Hubert's suggested solution EVP_MD_do_all_provided() worked almost straight forward. The function signature is a bit different and I got "undefined" in the result set. Filtering out NID_undef got right of it.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Mar 23, 2022 

    $ ./python Tools/ssl/multissltests.py --openssl 3.0.2 --steps modules
$ ./python -c "import hashlib; print(hashlib.algorithms_available)"
{'blake2b', 'sha512', 'sm3', 'shake_128', 'md5', 'sha3_256', 'sha224', 'sha512_224', 'sha3_384', 'sha384', 'md5-sha1', 'sha3_224', 'shake_256', 'sha3_512', 'sha512_256', 'sha1', 'sha256', 'blake2s'}

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Mar 23, 2022

    New changeset 48e2010 by Christian Heimes in branch 'main':
    bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076)
    48e2010

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Mar 23, 2022

    New changeset ec3589f by Miss Islington (bot) in branch '3.9':
    bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076)
    ec3589f

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Mar 23, 2022

    New changeset 1b6acaa by Christian Heimes in branch '3.10':
    [3.10] bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076) (GH-32085)
    1b6acaa

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @kumaraditya303
    Copy link
    Contributor

    @kumaraditya303 kumaraditya303 commented May 9, 2022

    @tiran can this be closed now ?

    @panlinux panlinux mentioned this issue May 30, 2022
    Open
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 3.10 3.11 expert-SSL extension-modules stdlib type-bug
    Projects
    OpenSSL 3
    Status: Todo
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @tiran @erlend-aasland @miss-islington @kumaraditya303