Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Python compatible with OpenSSL 3.0.0 #83001

Closed
tiran opened this issue Nov 16, 2019 · 29 comments
Closed

Make Python compatible with OpenSSL 3.0.0 #83001

tiran opened this issue Nov 16, 2019 · 29 comments
Assignees
@tiran
Labels
3.9 3.10 3.11 expert-SSL type-feature

Comments

@tiran
Copy link
Member

@tiran tiran commented Nov 16, 2019

BPO 38820
Nosy @tiran, @mcepl, @ambv, @stratakis, @hroncok, @miss-islington, @iritkatriel
PRs
  • #17190
  • #17499
  • #17500
  • #25316
  • #25319
  • #25320
  • #25321
  • #25537
  • #25624
  • #25626
  • #25627
  • #25817
  • #25942
  • #25943
  • #25944
  • #26266
  • #26269
  • #26769
  • #26799
  • #28205
  • #28216
  • #28217
    		•
    Dependencies
  • bpo-43788: OpenSSL 3.0.0: Make ssl_data.h version specific
  • bpo-43789: OpenSSL 3.0.0: password callback called multiple times
  • bpo-43791: OpenSSL 3.0.0: TLS 1.0 / 1.1 connections fail with TLSV1_ALERT_INTERNAL_ERROR
  • bpo-43794: OpenSSL 3.0.0: Handle UNEXPECTED_EOF_WHILE_READING / wrap SSL_OP_IGNORE_UNEXPECTED_EOF
  • bpo-43799: OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1
  • bpo-43811: Run GHA CI with multiple OpenSSL versions
  • bpo-43920: OpenSSL 3.0.0: handle empty cadata consistently
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tiran'
closed_at = None
created_at = <Date 2019-11-16.15:06:59.538>
labels = ['expert-SSL', 'type-feature', '3.9', '3.10', '3.11']
title = 'Make Python compatible with OpenSSL 3.0.0'
updated_at = <Date 2021-09-08.17:05:14.504>
user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2021-09-08.17:05:14.504>
actor = 'lukasz.langa'
assignee = 'christian.heimes'
closed = False
closed_date = None
closer = None
components = ['SSL']
creation = <Date 2019-11-16.15:06:59.538>
creator = 'christian.heimes'
dependencies = ['43788', '43789', '43791', '43794', '43799', '43811', '43920']
files = []
hgrepos = []
issue_num = 38820
keywords = ['patch']
message_count = 27.0
messages = ['356750', '356759', '357979', '357980', '357981', '378790', '378807', '390485', '390650', '390662', '390664', '390666', '391688', '391887', '391892', '391940', '392701', '393098', '394031', '396120', '396123', '397326', '401311', '401355', '401407', '401408', '401409']
nosy_count = 8.0
nosy_names = ['christian.heimes', 'mcepl', 'lukasz.langa', 'cstratak', 'hroncok', 'miss-islington', 'iritkatriel', 'bweeks']
pr_nums = ['17190', '17499', '17500', '25316', '25319', '25320', '25321', '25537', '25624', '25626', '25627', '25817', '25942', '25943', '25944', '26266', '26269', '26769', '26799', '28205', '28216', '28217']
priority = 'high'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38820'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 16, 2019

    OpenSSL 3.0.0 is currently development [1]. I'm expecting a first beta release in December. Final release is scheduled for Q2 2020. OpenSSL 3.0.0 is API and feature compatible to OpenSSL 1.1.0 and 1.1.1. Only minor changes are required:

    • OpenSSL version number is >= 3.0.0, which breaks test_openssl_version
    • GENERAL_NAME_print() no longer adds trailing newline to IPv6 address strings.
    • ERR_func_error_string is deprecated

    [1] https://www.openssl.org/blog/blog/2019/11/07/3.0-update/

    @tiran tiran self-assigned this Nov 16, 2019
    @tiran tiran added expert-SSL type-feature labels Nov 16, 2019
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 16, 2019

    PR #61392 fixes test_openssl_version and removes the trailing newline from IPv6 addresses on all OpenSSL versions. I prefer to have the output consistent on all OpenSSL versions. The newline was silly any way.

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Dec 7, 2019

    New changeset 2b7de66 by Miss Islington (bot) (Christian Heimes) in branch 'master':
    bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190)
    2b7de66

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Dec 7, 2019

    New changeset 9d3cacd by Miss Islington (bot) in branch '3.8':
    [3.8] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17499)
    9d3cacd

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Dec 7, 2019

    New changeset a197f8a by Miss Islington (bot) in branch '3.7':
    [3.7] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17500)
    a197f8a

    @iritkatriel
    Copy link
    Member

    @iritkatriel iritkatriel commented Oct 16, 2020

    Can this be closed?

    @methane methane closed this as completed Oct 17, 2020
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 17, 2020

    No, this is still work in progress.

    @tiran tiran reopened this Oct 17, 2020
    @hroncok
    Copy link
    Mannequin

    @hroncok hroncok mannequin commented Apr 7, 2021

    Python 3.10.0a7 with OpenSSL 3.0 from https://copr.fedorainfracloud.org/coprs/saprasad/openssl-3.0/ in https://copr.fedorainfracloud.org/coprs/g/python/openssl-3.0/package/python3.10/ (full logs available there).

    3 tests failed:
    test_imaplib test_ssl test_urllib2_localnet

    Many:

    ssl.SSLError: [SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:2628)

    Also:

    Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.10.0a7/Lib/test/test_ssl.py", line 1413, in test_load_cert_chain
    ctx.load_cert_chain(CERTFILE_PROTECTED, password=getpass_huge)
SystemError: _PyEval_EvalFrameDefault returned a result with an exception set

    And:

    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1122)

    ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:1122)

    ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1122)

    @hroncok hroncok mannequin added the 3.10 label Apr 7, 2021
    @tiran tiran removed the 3.7 label Apr 9, 2021
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Apr 9, 2021

    Miro,

    I have pushed several fixes for OpenSSL 3.0.0

    • bpo-43788 addresses wrong library and error reason codes (e.g. KRB5_S_TKT_NYV)
    • bpo-43789 fixes an issue with exception state in password callbacks (_PyEval_EvalFrameDefault returned a result with an exception set)
    • bpo-43791 disables TLS 1.0 and 1.1 testing with OpenSSL 3.0.0. I'll have to talk to upstream and figure out a better solution.
    • bpo-43794 adds OP_IGNORE_UNEXPECTED_EOF and sets it by default. This makes the code behave like OpenSSL 1.1.0 and 1.0.2.

    I'll look into the other issues next week.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Apr 9, 2021

    New changeset 2d7fdc9 by Christian Heimes in branch 'master':
    bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
    2d7fdc9

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Apr 9, 2021

    New changeset ffb05bb by Miss Islington (bot) in branch '3.8':
    bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
    ffb05bb

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Apr 9, 2021

    New changeset 7c8796a by Miss Islington (bot) in branch '3.9':
    bpo-38820: OpenSSL 3.0.0: Use supported hashing algos in doc test (GH-25319)
    7c8796a

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Apr 23, 2021

    New changeset dcf6581 by Christian Heimes in branch 'master':
    bpo-38820: Test with OpenSSL 3.0.0-alpha15 (GH-25537)
    dcf6581

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Apr 26, 2021

    New changeset 3c586ca by Christian Heimes in branch 'master':
    bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624)
    3c586ca

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Apr 26, 2021

    New changeset 10ee266 by Miss Islington (bot) in branch '3.8':
    [3.8] bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) (GH-25627)
    10ee266

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Apr 26, 2021

    New changeset 3b917d1 by Miss Islington (bot) in branch '3.9':
    [3.9] bpo-38820: Old OpenSSL 3.0.0 releases are in /old/3.0/ (GH-25624) (GH-25626)
    3b917d1

    @tiran
    Copy link
    Member Author

    @tiran tiran commented May 2, 2021

    New changeset d8389e3 by Christian Heimes in branch 'master':
    bpo-38820: Add ssl, hashlib, and hmac changes to whatsnew 3.10 (GH-25817)
    d8389e3

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented May 6, 2021

    New changeset f8778f9 by Miss Islington (bot) in branch '3.10':
    bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942)
    f8778f9

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented May 20, 2021

    New changeset 36843f7 by Miss Islington (bot) in branch '3.10':
    bpo-38820: Test with OpenSSL 3.0.0-alpha17 (GH-26266)
    36843f7

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Jun 19, 2021

    New changeset 44fb551 by Christian Heimes in branch 'main':
    bpo-38820: Test with OpenSSL 3.0.0-beta1 (GH-26769)
    44fb551

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 19, 2021

    New changeset c6cd2ec by Miss Islington (bot) in branch '3.10':
    [3.10] bpo-38820: Test with OpenSSL 3.0.0-beta1 (GH-26769) (GH-26799)
    c6cd2ec

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Jul 12, 2021

    New changeset c92b391 by Christian Heimes in branch '3.9':
    [3.9] bpo-38820: Test with OpenSSL 3.0.0-alpha16 (GH-25942) (bpo-25944)
    c92b391

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Sep 7, 2021

    New changeset cc7c680 by Christian Heimes in branch 'main':
    bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205)
    cc7c680

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Sep 8, 2021

    New changeset 2fe15db by Miss Islington (bot) in branch '3.10':
    bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205)
    2fe15db

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Sep 8, 2021

    New changeset 7a6178a by Łukasz Langa in branch '3.9':
    [3.9] bpo-38820: Test with OpenSSL 3.0.0 final (GH-28205) (GH-28217)
    7a6178a

    @ambv
    Copy link
    Contributor

    @ambv ambv commented Sep 8, 2021

    Christian, Python is now tested with 3.0.0 final in 3.9, 3.10, and 3.11. Looks like we can close this!

    Thank you for this big body of work ✨ 🍰 ✨

    @ambv ambv added 3.11 and removed 3.8 labels Sep 8, 2021
    @ambv
    Copy link
    Contributor

    @ambv ambv commented Sep 8, 2021

    (I'll let you close this yourself when you determine that the two remaining open dependencies can be closed as well.)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @Steve123p
    Copy link

    @Steve123p Steve123p commented Apr 11, 2022

    Is this going to be officially closed?

    @berghaus berghaus mentioned this issue May 3, 2022
    Open
    @kumaraditya303
    Copy link
    Contributor

    @kumaraditya303 kumaraditya303 commented May 9, 2022

    I am closing this as it is fixed, if required it can reopened.

    @native-api native-api mentioned this issue May 11, 2022
    Closed
    11 tasks
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @tiran tiran

    Labels
    3.9 3.10 3.11 expert-SSL type-feature
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    7 participants
    @ambv @methane @tiran @iritkatriel @miss-islington @kumaraditya303 @Steve123p