Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[2.7] bpo-28043: improved default settings for SSLContext #10608

Merged
merged 2 commits into from
Feb 15, 2019
Merged

[2.7] bpo-28043: improved default settings for SSLContext #10608

merged 2 commits into from
Feb 15, 2019

Conversation

stratakis
Copy link
Contributor

@stratakis stratakis commented Nov 20, 2018 •
edited by bedevere-bot
Loading

This is a backport of that change for the 2.7 branch.

It is made on top of #10607

https://bugs.python.org/issue28043

@stratakis
Copy link
Contributor Author

All tests pass when utilizing multissltests.py

@encukou encukou requested a review from tiran December 3, 2018 13:08
@vstinner
Copy link
Member

Please mention commit 358cfd4 in the commit message of the 2nd commit of this PR.

@vstinner
Copy link
Member

test_ssl fails on my Fedora 29:

ERROR: test_tls1_3 (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/2.7/Lib/test/test_ssl.py", line 2848, in test_tls1_3
    with context.wrap_socket(socket.socket()) as s:
AttributeError: __exit__

======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/2.7/Lib/test/test_ssl.py", line 810, in test_options
    self.assertEqual(default, ctx.options)
AssertionError: 2185363540 != 2186412116L

======================================================================
FAIL: test_default_ecdh_curve (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/2.7/Lib/test/test_ssl.py", line 2871, in test_default_ecdh_curve
    self.assertIn("ECDH", s.cipher()[0])
AssertionError: 'ECDH' not found in 'TLS_AES_256_GCM_SHA384'

@vstinner
Copy link
Member 

ERROR: test_tls1_3 (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/2.7/Lib/test/test_ssl.py", line 2848, in test_tls1_3
    with context.wrap_socket(socket.socket()) as s:
AttributeError: __exit__

This one is an obvious bug in your backport, following change fix it:

diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
index 9e9744f675..83ba7e0de2 100644
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -2845,7 +2847,8 @@ else:
                 ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1_2
             )
             with ThreadedEchoServer(context=context) as server:
-                with context.wrap_socket(socket.socket()) as s:
+                s = context.wrap_socket(socket.socket())
+                with closing(s):
                     s.connect((HOST, server.port))
                     self.assertIn(s.cipher()[0], [
                         'TLS_AES_256_GCM_SHA384',

Next:

FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/2.7/Lib/test/test_ssl.py", line 810, in test_options
    self.assertEqual(default, ctx.options)
AssertionError: 2185363540 != 2186412116L

This test requires OP_ENABLE_MIDDLEBOX_COMPAT which has been added by commit 05d9fe3. Minimum patch fixing the test:

diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
index 9e9744f675..83ba7e0de2 100644
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -82,6 +82,7 @@ OP_NO_COMPRESSION = getattr(ssl, "OP_NO_COMPRESSION", 0)
 OP_SINGLE_DH_USE = getattr(ssl, "OP_SINGLE_DH_USE", 0)
 OP_SINGLE_ECDH_USE = getattr(ssl, "OP_SINGLE_ECDH_USE", 0)
 OP_CIPHER_SERVER_PREFERENCE = getattr(ssl, "OP_CIPHER_SERVER_PREFERENCE", 0)
+OP_ENABLE_MIDDLEBOX_COMPAT = getattr(ssl, "OP_ENABLE_MIDDLEBOX_COMPAT", 0)
 
 
 def handle_error(prefix):
@@ -806,7 +807,8 @@ class ContextTests(unittest.TestCase):
         default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3)
         # SSLContext also enables these by default
         default |= (OP_NO_COMPRESSION | OP_CIPHER_SERVER_PREFERENCE |
-                    OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE)
+                    OP_SINGLE_DH_USE | OP_SINGLE_ECDH_USE |
+                    OP_ENABLE_MIDDLEBOX_COMPAT)
         self.assertEqual(default, ctx.options)
         ctx.options |= ssl.OP_NO_TLSv1
         self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options)
diff --git a/Modules/_ssl.c b/Modules/_ssl.c
index 80078aa3cb..93b635cc4a 100644
--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -4411,6 +4411,10 @@ init_ssl(void)
     PyModule_AddIntConstant(m, "OP_NO_COMPRESSION",
                             SSL_OP_NO_COMPRESSION);
 #endif
+#ifdef SSL_OP_ENABLE_MIDDLEBOX_COMPAT
+    PyModule_AddIntConstant(m, "OP_ENABLE_MIDDLEBOX_COMPAT",
+                            SSL_OP_ENABLE_MIDDLEBOX_COMPAT);
+#endif
 
 #if HAVE_SNI
     r = Py_True;

And the last one:

FAIL: test_default_ecdh_curve (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/2.7/Lib/test/test_ssl.py", line 2871, in test_default_ecdh_curve
    self.assertIn("ECDH", s.cipher()[0])
AssertionError: 'ECDH' not found in 'TLS_AES_256_GCM_SHA384'

this test has been fixed in master by:

diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
index 9e9744f675..83ba7e0de2 100644
--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -2859,6 +2862,9 @@ else:
             # should be enabled by default on SSL contexts.
             context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
             context.load_cert_chain(CERTFILE)
+            # TLSv1.3 defaults to PFS key agreement and no longer has KEA in
+            # cipher name.
+            context.options |= ssl.OP_NO_TLSv1_3
             # Prior to OpenSSL 1.0.0, ECDH ciphers have to be enabled
             # explicitly using the 'ECCdraft' cipher alias.  Otherwise,
             # our default cipher list should prefer ECDH-based ciphers

This change is extract from commit cb5b68a.

vstinner
vstinner reviewed Feb 14, 2019
View reviewed changes
Doc/library/ssl.rst Outdated
@@ -1058,6 +1058,17 @@ to speed up repeated connections from the same clients.
:func:`create_default_context` lets the :mod:`ssl` module choose
security settings for a given purpose.

.. versionchanged:: 2.7
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version should be 2.7.16.

@tiran @stratakis
bpo-28043: SSLContext has improved default settings
2050471 
The options OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE,
OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE, OP_NO_SSLv2 (except
for PROTOCOL_SSLv2), and OP_NO_SSLv3 (except for PROTOCOL_SSLv3)
are set by default. The initial cipher suite list contains only
HIGH ciphers, no NULL ciphers and MD5 ciphers (except for PROTOCOL_SSLv2).

(cherry picked from commit 358cfd4)
@stratakis stratakis force-pushed the 2.7-ssl-improved-default branch from 5224adf to 2050471 Compare February 15, 2019 13:55
@vstinner vstinner merged commit b8eaec6 into python:2.7 Feb 15, 2019
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x SLES 2.7 has failed when building commit b8eaec6.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/66/builds/338) and take a look at the build logs.
  4. Check if the failure is related to this commit (b8eaec6) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/66/builds/338

Click to see traceback logs 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named test.test_support


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 188, in test_custom_headers
    opener.open(request)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_fileno (test.test_urllib2net.OtherNetworkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 174, in test_fileno
    res = opener.open(req)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_close (test.test_urllib2net.CloseSocketTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 88, in test_close
    response = _urlopen_with_retry("http://www.example.com/")
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_basic (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 263, in test_http_basic
    u = _urlopen_with_retry(url)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_default_timeout (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 283, in test_http_no_timeout
    u = _urlopen_with_retry(url, timeout=None)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_timeout (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 291, in test_http_timeout
    u = _urlopen_with_retry(url, timeout=120)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

----------------------------------------------------------------------
Ran 15 tests in 1.033s

FAILED (errors=7, skipped=1)
Re-running test 'test_urllibnet' in verbose mode


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/test/test_urllibnet.py", line 202, in test_data_header
    time.strptime(datevalue, dateformat)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/_strptime.py", line 478, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/home/dje/cpython-buildarea/2.7.edelsohn-sles-z/build/Lib/_strptime.py", line 329, in _strptime
    found = format_regex.match(data_string)
TypeError: expected string or buffer

----------------------------------------------------------------------
Ran 13 tests in 0.111s

FAILED (errors=1)

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Debian 2.7 has failed when building commit b8eaec6.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/55/builds/355) and take a look at the build logs.
  4. Check if the failure is related to this commit (b8eaec6) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/55/builds/355

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllibnet.py", line 202, in test_data_header
    time.strptime(datevalue, dateformat)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/_strptime.py", line 478, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/_strptime.py", line 329, in _strptime
    found = format_regex.match(data_string)
TypeError: expected string or buffer

----------------------------------------------------------------------
Ran 13 tests in 0.131s

FAILED (errors=1)
Re-running test 'test_urllib2net' in verbose mode


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 188, in test_custom_headers
    opener.open(request)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_fileno (test.test_urllib2net.OtherNetworkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 174, in test_fileno
    res = opener.open(req)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_close (test.test_urllib2net.CloseSocketTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 88, in test_close
    response = _urlopen_with_retry("http://www.example.com/")
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_basic (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 263, in test_http_basic
    u = _urlopen_with_retry(url)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_default_timeout (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 283, in test_http_no_timeout
    u = _urlopen_with_retry(url, timeout=None)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_timeout (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 291, in test_http_timeout
    u = _urlopen_with_retry(url, timeout=120)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-debian-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

----------------------------------------------------------------------
Ran 15 tests in 1.030s

FAILED (errors=7, skipped=1)

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL 2.7 has failed when building commit b8eaec6.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/25/builds/343) and take a look at the build logs.
  4. Check if the failure is related to this commit (b8eaec6) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/25/builds/343

Click to see traceback logs 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python2.7/test/test_support.py", line 22, in <module>
    import _testcapi
ImportError: No module named _testcapi


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllibnet.py", line 202, in test_data_header
    time.strptime(datevalue, dateformat)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/_strptime.py", line 478, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/_strptime.py", line 329, in _strptime
    found = format_regex.match(data_string)
TypeError: expected string or buffer

----------------------------------------------------------------------
Ran 13 tests in 5.148s

FAILED (errors=1)
Re-running test 'test_urllib2net' in verbose mode


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 188, in test_custom_headers
    opener.open(request)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_fileno (test.test_urllib2net.OtherNetworkTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 174, in test_fileno
    res = opener.open(req)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_close (test.test_urllib2net.CloseSocketTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 88, in test_close
    response = _urlopen_with_retry("http://www.example.com/")
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_basic (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 263, in test_http_basic
    u = _urlopen_with_retry(url)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_default_timeout (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 283, in test_http_no_timeout
    u = _urlopen_with_retry(url, timeout=None)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

======================================================================
ERROR: test_http_timeout (test.test_urllib2net.TimeoutTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 291, in test_http_timeout
    u = _urlopen_with_retry(url, timeout=120)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 25, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/home/dje/cpython-buildarea/2.7.edelsohn-rhel-z/build/Lib/test/test_urllib2net.py", line 21, in _retry_thrice
    raise last_exc
HTTPError: HTTP Error 403: Forbidden

----------------------------------------------------------------------
Ran 15 tests in 1.035s

FAILED (errors=7, skipped=1)

@vstinner
Copy link
Member

You can ignore buildbots failures at this point. I'm working right now with @stratakis to push more fixes for OpenSSL 1.1.1.

@vstinner
Copy link
Member

https://buildbot.python.org/all/#/builders/25/builds/343

ERROR: test_data_header (test.test_urllibnet.urlretrieveNetworkTests)
ERROR: test_custom_headers (test.test_urllib2net.OtherNetworkTests)
ERROR: test_fileno (test.test_urllib2net.OtherNetworkTests)
ERROR: test_close (test.test_urllib2net.CloseSocketTest)
ERROR: test_http_basic (test.test_urllib2net.TimeoutTest)
ERROR: test_http_default_timeout (test.test_urllib2net.TimeoutTest)
ERROR: test_http_no_timeout (test.test_urllib2net.TimeoutTest)
ERROR: test_http_timeout (test.test_urllib2net.TimeoutTest)

I checked all these tests: they all connect to http://www.example.com : it's clear-text HTTP, it doesn't use SSL. I also checked manually that the website doesn't redirect to HTTPS. So these failures are just random network errors.

https://buildbot.python.org/all/#/builders/55/builds/355
https://buildbot.python.org/all/#/builders/66/builds/338

These buildbot workers are also s390x workers owned by David Edelsohn, so likely running in the same network and had the same network issues.

--

I tested manually test_urllib2net: it pass as expected.

@csabella csabella mentioned this pull request May 29, 2019
Closed
@stratakis stratakis deleted the 2.7-ssl-improved-default branch June 18, 2020 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vstinner vstinner vstinner left review comments

@tiran tiran Awaiting requested review from tiran

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@stratakis @vstinner @bedevere-bot @the-knights-who-say-ni @tiran