python / cpython

(cherry picked from commit f4800b8)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>

…17810)

(cherry picked from commit 946b29e)

Co-authored-by: Benjamin Peterson <benjamin@python.org>

…acOS Catalina (GH-17636) (GH-17638)

(cherry picked from commit bf3aa10)

Co-authored-by: Ned Deily <nad@python.org>

…nt()'s *reuse_address* parameter (GH-17595). (GH-17632)

(cherry picked from commit f501db2)

Co-authored-by: Kyle Stanley <aeros167@gmail.com>

Co-authored-by: tirkarthi
(cherry picked from commit 1988344)

Co-authored-by: Kyle Stanley <aeros167@gmail.com>

…USEADDR (GH-17311). (GH-17571)

(cherry picked from commit ab513a3)

Co-authored-by: Kyle Stanley <aeros167@gmail.com>

…put format (GH-17418) (GH-17444)

(cherry picked from commit a62ad47)

Co-authored-by: Matthew Rollings <1211162+stealthcopter@users.noreply.github.com>

The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).

LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.

The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to

    \d+-\w+-\d+(\s*\s*\s*)$

Therefore, a long sequence of spaces can trigger bad performance.

Matching a malicious string such as

    LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")

caused catastrophic backtracking.

The fix removes ambiguity about which \s* should match a particular
space.

You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.

    from http.server import BaseHTTPRequestHandler, HTTPServer

    def make_set_cookie_value(n_spaces):
        spaces = " " * n_spaces
        expiry = f"1-c-1{spaces}!"
        return f"b;Expires={expiry}"

    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            self.log_request(204)
            self.send_response_only(204)  GH- Don't bother sending Server and Date
            n_spaces = (
                int(self.path[1:])  GH- Can GET e.g. /100 to test shorter sequences
                if len(self.path) > 1 else
                65506  GH- Max header line length 65536
            )
            value = make_set_cookie_value(n_spaces)
            for i in range(99):  GH- Not necessary, but we can have up to 100 header lines
                self.send_header("Set-Cookie", value)
            self.end_headers()

    if __name__ == "__main__":
        HTTPServer(("", 44020), Handler).serve_forever()

This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.

Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :

    import http.cookiejar, urllib.request
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://localhost:44020/")

The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):

    import requests
    requests.get("http://localhost:44020/")

* Regression test for http.cookiejar REDoS

If we regress, this test will take a very long time.

* Improve performance of http.cookiejar.ISO_DATE_RE

A string like

"444444" + (" " * 2000) + "A"

could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
(cherry picked from commit 1b779bf)

Co-authored-by: bcaller <bcaller@users.noreply.github.com>

(cherry picked from commit 01659ca)

Co-authored-by: Ned Deily <nad@python.org>

(cherry picked from commit dfe726b)

Co-authored-by: Kyle Stanley <aeros167@gmail.com>

(cherry picked from commit 3f36043)

Co-authored-by: Ned Deily <nad@python.org>

(cherry picked from commit 4504b45)

Co-authored-by: Julien Palard <julien@palard.fr>

…alidation and encoding behavior (GH-16448) (GH-16462)

(cherry picked from commit 7774d78)

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>

Escape the server title of xmlrpc.server.DocXMLRPCServer
when rendering the document page as HTML.

(cherry picked from commit e8650a4)

…6410)

Fixes CVE-2019-15903. See full changelog at https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes..
(cherry picked from commit 52b9408)

Before:

        >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses
        (Address(display_name='', username='a', domain='malicious.org'),)

        >>> parseaddr('a@malicious.org@important.com')
        ('', 'a@malicious.org')

    After:

        >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses
        (Address(display_name='', username='', domain=''),)

        >>> parseaddr('a@malicious.org@important.com')
        ('', 'a@')

https://bugs.python.org/issue34155
(cherry picked from commit 8cb65d1)

Co-authored-by: jpic <jpic@users.noreply.github.com>

…aders (GH-14794) (GH-14817)

Some crafted email header would cause the get_parameter method to run in an
infinite loop causing a DoS attack surface when parsing those headers. This
patch fixes that by making sure the DQUOTE character is handled to prevent
going into an infinite loop.
(cherry picked from commit a4a994b)

Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>

As far as I can tell, this infinite loop would be triggered if:

1. The value being folded contains a single word (no spaces) longer than
   max_line_length
2. The max_line_length is shorter than the encoding's name + 9
   characters.

bpo-36564: https://bugs.python.org/issue36564
(cherry picked from commit f69d5c6)

Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>

Also fix a name misspelling.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

… (GH-14549)

Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS.  As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems.  Reverting the change to the interpreter stack size at link
time helped for release builds but caused some tests to fail when
built --with-pydebug.  Try the opposite approach: continue to build
the interpreter with an increased stack size on macOS and remove
the failing setrlimit call in regrtest initialization.  This will
definitely avoid the resource.RLIMIT_STACK error and should have
no, or fewer, side effects.
(cherry picked from commit 5bbbc73)

Co-authored-by: Ned Deily <nad@python.org>

(cherry picked from commit 2cd0792)

Co-authored-by: Benjamin Peterson <benjamin@python.org>

GH-14472)

(cherry picked from commit 95da310)

Co-authored-by: Benjamin Peterson <benjamin@python.org>

(cherry picked from commit 3b03b09)

Co-authored-by: Benjamin Peterson <benjamin@python.org>