…n. (GH-16950) (GH-16999)

Replace all *NULL* with ``NULL``.
(cherry picked from commit 25fc088)

Also fix some other formatting.
(cherry picked from commit e835b31)

Change the url from docs.python-requests.org to requests.readthedocs.io
(cherry picked from commit 112f2b8)

Co-authored-by: Simon Legner <Simon.Legner@gmail.com>

(cherry picked from commit 65c7382)

Co-authored-by: MaT1g3R <peijun.ma@protonmail.com>

…H-16799) (GH-17016)

(cherry picked from commit 79d4ed1)

* Add missing test class (mistake in GH-4455)

* Increase coverage with 4 more test cases

* Rename neg_uid to huge_uid in test_modified_uid_huge

* Replace test_main() with unittest.main()

* Update plistlib docs.
(cherry picked from commit d0d9f7c)

Co-authored-by: Jon Janzen <jjjonjanzen@gmail.com>

Whenever I use `path.suffix` I have to check again whether it includes the dot or not. I decided to add it to the docstring so I won't have to keep checking.

https://bugs.python.org/issue38422

Automerge-Triggered-By: @pitrou
(cherry picked from commit 8d4fef4)

Co-authored-by: Ram Rachum <ram@rachum.com>

(cherry picked from commit d0e0f5b)

Co-authored-by: Dima Tisnek <dimaqq@gmail.com>

(cherry picked from commit 1cdadf4)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>

s/pathing/patching/
(cherry picked from commit 25fa3ec)

Co-authored-by: Michael Haas <micha2718l@gmail.com>

…acros in ``pythonrun.h``. (GH-17056)

(cherry picked from commit 62161ce)

Co-authored-by: Benjamin Peterson <benjamin@python.org>

(cherry picked from commit 99b7701)

Co-authored-by: Борис Верховский <boris.verk@gmail.com>

GH-17026)

This was never intented to be called manually from PyInit_*.

Also, clarify PyState_RemoveModule return value.
(cherry picked from commit 9bc94ec)


Co-authored-by: Petr Viktorin <encukou@gmail.com>


https://bugs.python.org/issue38159



Automerge-Triggered-By: @encukou

Fixed what seemed to be a weird phrasing.

(cherry picked from commit 5e01a65)

Co-authored-by: Jules Lasne (jlasne) <jules.lasne@gmail.com>

…7043)

(cherry picked from commit 6552563)

Co-authored-by: Alexandru Ardelean <ardeleanalex@gmail.com>

(cherry picked from commit 56698d5)

Co-authored-by: Ammar Askar <ammar@ammaraskar.com>

…17078)

(cherry picked from commit 7f46049)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>

(cherry picked from commit befa032)

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

(cherry picked from commit 98480ce)

Co-authored-by: Jonathan Scholbach <j.scholbach@posteo.de>

…7111) (GH-17113)

(cherry picked from commit 733b9a3)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>

…H-17103)

(cherry picked from commit c8b53dc)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

Updates documentation around email.utils.parsedate_tz().

Currently, the documentation specifies that when a string without a is timezone passed to parsedate_tz(), the last tuple is returned as ```None```.

This is no longer true since Python 3.3


https://bugs.python.org/issue38421
(cherry picked from commit a12255d)


Co-authored-by: David K <dave@paddez.com>


https://bugs.python.org/issue38421



Automerge-Triggered-By: @encukou

* "Return true/false" is replaced with "Return ``True``/``False``"
  if the function actually returns a bool.
* Fixed formatting of some True and False literals (now in monospace).
* Replaced "True/False" with "true/false" if it can be not only bool.
* Replaced some 1/0 with True/False if it corresponds the code.
* "Returns <bool>" is replaced with "Return <bool>".
(cherry picked from commit 138ccbb)

…GH-17127)

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit d593881)

Co-authored-by: jsnklln <jsnklln@gmail.com>

…mode files. (GH-17136)

This change, which follows the behavior of C stdio's fdopen and Python 2's file object, allows pipes to be opened in append mode..
(cherry picked from commit 74fa9f7)

Contributions on bpo-26467 and bpo-29302.
(cherry picked from commit d6d6e2a)

Co-authored-by: Ilya Kulakov <kulakov.ilya@gmail.com>

(cherry picked from commit 2d56af7)

Co-authored-by: Shu <23287722+susan-shu-c@users.noreply.github.com>

This immediately toggles shell, editor, and output windows, but does not affect other input widgets.
(cherry picked from commit 9c28449)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

if parent `__init__` is not called from a constructor of object derived from `asyncio.Future`

https://bugs.python.org/issue38785
(cherry picked from commit dad6be5)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>

…H-17162)

(cherry picked from commit e8acc86)

Co-authored-by: Andrey Doroschenko <dorosch.github.io@yandex.ru>

…H-17039)

(cherry picked from commit b44ffc8)

Co-authored-by: Marco Paolini <mpaolini@users.noreply.github.com>

…GH-17176)

The C-API docs are a bit sparse on the interplay between C `fork()` and the CPython runtime.  This change adds some more information on the subject.

https://bugs.python.org/issue38816
(cherry picked from commit 73cdb0c)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>

…H-17123)

Small docs update for [bpo-34651](https://bugs.python.org/issue34651).

Other references to fork (e.g. the PyOS.*Fork functions or discussions of fork() when embedding Python) point back to os.fork, so I don't think any other updates are needed.

https://bugs.python.org/issue38778

Automerge-Triggered-By: @ericsnowcurrently
(cherry picked from commit b220300)

Co-authored-by: Phil Connell <pconnell@gmail.com>

(cherry picked from commit 84f2528)

Co-authored-by: HongWeipeng <961365124@qq.com>

(cherry picked from commit 0fe0b88)

Co-authored-by: Jules Lasne (jlasne) <jules.lasne@gmail.com>

…H-16967)

Ensure isabs() is always True for \\?\ prefixed paths
Avoid unnecessary usage of readlink() to avoid resolving broken links incorrectly
Ensure shutil tests run in test directory

(cherry picked from commit 7c6130c)

Co-authored-by: Steve Dower <steve.dower@python.org>

changed 'This is bad class design, but save some typing'
into 'This is bad class design, but saves some typing'.
(cherry picked from commit d0acdfc)

Co-authored-by: Jason (Perry) Taylor <jtaylor@seek.com.au>

https://bugs.python.org/issue38823
(cherry picked from commit c3f6bdc)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>

https://bugs.python.org/issue38823
(cherry picked from commit 143a97f)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>

(cherry picked from commit 4544e78)

Co-authored-by: alclarks <57201106+alclarks@users.noreply.github.com>

(cherry picked from commit 04c79d6)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>

…-17221)

(cherry picked from commit 5383956)

…-17097) (GH-17223)

(cherry picked from commit 1062715)

https://bugs.python.org/issue38823
(cherry picked from commit 289cf0f)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>

…7065)

`~` is now treated as an unreserved character (i.e. it doesn't get quoted), not a reserved one.
(cherry picked from commit f49f6ba)

Co-authored-by: Роман Донченко <dpb@corrigendum.ru>

…-17164)

https://bugs.python.org/issue38809
(cherry picked from commit ee703cb)

Co-authored-by: Tal Einat <taleinat+github@gmail.com>

(cherry picked from commit 00923c6)

Co-authored-by: Steve Dower <steve.dower@python.org>

https://bugs.python.org/issue38722

Automerge-Triggered-By: @taleinat
(cherry picked from commit e243bae)

Co-authored-by: jsnklln <jsnklln@gmail.com>

… single argument (GH-17243)

(cherry picked from commit dcf1f83)

Co-authored-by: Steve Dower <steve.dower@python.org>

…types (GH-17160) (GH-17249)

(cherry picked from commit fe75b62)

Co-authored-by: Tomás Farías <tomasfariassantana@gmail.com>

Automerge-Triggered-By: @csabella
(cherry picked from commit b1f160a)

Co-authored-by: Jules Lasne (jlasne) <jules.lasne@gmail.com>

Ignore `GeneratorExit` exceptions when throwing an exception into the `aclose` coroutine of an asynchronous generator.

https://bugs.python.org/issue35409
(cherry picked from commit 8e0de2a)

Co-authored-by: Vincent Michel <vxgmichel@gmail.com>

…7088)

This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`.

In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own).

This change forces the Process object to update its `native_id` attribute during the bootstrap process.

cc @vstinner

https://bugs.python.org/issue38707

Automerge-Triggered-By: @pitrou
(cherry picked from commit c6b20be)

Co-authored-by: Jake Tesler <jake.tesler@gmail.com>

…H-17250)

(cherry picked from commit ac22354)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>

These Format menu functions (default shortcuts Alt-T and Alt-U)
were mistakenly disabled in 3.7.5 and 3.8.0.
(cherry picked from commit b846247)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

(cherry picked from commit 33b671e)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>

(cherry picked from commit d51a363)

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>

(cherry picked from commit be5c79e)

Co-authored-by: Federico Bond <federicobond@gmail.com>

…nix (GH-17294)

on platforms lacking a functional bind() for named unix domain sockets

https://bugs.python.org/issue38841

Automerge-Triggered-By: @asvetlov
(cherry picked from commit 559bad1)

Co-authored-by: xdegaye <xdegaye@gmail.com>

Fixed an incorrect sentence in Doc/c-api/mapping.rst I fell on
while translating the file.

skip issue

Automerge-Triggered-By: @csabella
(cherry picked from commit 06ca2a2)

Co-authored-by: Aveheuzed <a.masson555@ntymail.com>

Automerge-Triggered-By: @csabella
(cherry picked from commit 6db2fb7)

Co-authored-by: Jules Lasne (jlasne) <jules.lasne@gmail.com>

https://bugs.python.org/issue36277

Automerge-Triggered-By: @csabella
(cherry picked from commit 9391f6c)

Co-authored-by: Dave Nguyen <dv@dvnguyen.com>

This PR will make the following changes to the [_Built-in Functions_](https://docs.python.org/3/library/functions.html) chapter of the library documentation:

- improve hyperlinks in Sphinx roles (trailing 's' belong to hyperlinks).

Automerge-Triggered-By: @csabella
(cherry picked from commit d672791)

Co-authored-by: Géry Ogam <gery.ogam@gmail.com>

test_capi: trashcan tests now require the test "cpu" resource.
(cherry picked from commit 0127bb1)

Co-authored-by: Victor Stinner <vstinner@python.org>

GH-17126)

https://bugs.python.org/issue37838
(cherry picked from commit 0aca3a3)

Co-authored-by: benedwards14 <53377856+benedwards14@users.noreply.github.com>

… fix [3.8] (GH-17269) (#17304)

(cherry picked from commit 046442d)

Co-authored-by: Jason Fried <fried@fb.com>

(cherry picked from commit 65444cf)

Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>

Increment properly Py_True/Py_False reference counter for
_testcapi.WITH_PYMALLOC variable.

…-17252)

(cherry picked from commit 9960230)

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).

LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.

The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to

    \d+-\w+-\d+(\s*\s*\s*)$

Therefore, a long sequence of spaces can trigger bad performance.

Matching a malicious string such as

    LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")

caused catastrophic backtracking.

The fix removes ambiguity about which \s* should match a particular
space.

You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.

    from http.server import BaseHTTPRequestHandler, HTTPServer

    def make_set_cookie_value(n_spaces):
        spaces = " " * n_spaces
        expiry = f"1-c-1{spaces}!"
        return f"b;Expires={expiry}"

    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            self.log_request(204)
            self.send_response_only(204)  GH- Don't bother sending Server and Date
            n_spaces = (
                int(self.path[1:])  GH- Can GET e.g. /100 to test shorter sequences
                if len(self.path) > 1 else
                65506  GH- Max header line length 65536
            )
            value = make_set_cookie_value(n_spaces)
            for i in range(99):  GH- Not necessary, but we can have up to 100 header lines
                self.send_header("Set-Cookie", value)
            self.end_headers()

    if __name__ == "__main__":
        HTTPServer(("", 44020), Handler).serve_forever()

This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.

Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :

    import http.cookiejar, urllib.request
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://localhost:44020/")

The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):

    import requests
    requests.get("http://localhost:44020/")

* Regression test for http.cookiejar REDoS

If we regress, this test will take a very long time.

* Improve performance of http.cookiejar.ISO_DATE_RE

A string like

"444444" + (" " * 2000) + "A"

could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
(cherry picked from commit 1b779bf)

Co-authored-by: bcaller <bcaller@users.noreply.github.com>

The Y2K reference is not needed as it only points out that Python's use
of C standard functions doesn't generally suffer from Y2K issues; the
point regarding conventions for conversion of 2-digit years in
:func:`strptime` is still valid.
(cherry picked from commit 42bc60e)

Co-authored-by: Callum Ward <wards.callum@gmail.com>

* fix HTTP Digest handling in request.py

There is a bug triggered when server replies to a request with `WWW-Authenticate: Digest` where `qop="auth,auth-int"` rather than mere `qop="auth"`. Having both `auth` and `auth-int` is legitimate according to the `qop-options` rule in §3.2.1 of [[https://www.ietf.org/rfc/rfc2617.txt|RFC 2617]]:
>      qop-options       = "qop" "=" <"> 1GH-qop-value <">
>      qop-value         = "auth" | "auth-int" | token
> **qop-options**: [...] If present, it is a quoted string **of one or more** tokens indicating the "quality of protection" values supported by the server.  The value `"auth"` indicates authentication; the value `"auth-int"` indicates authentication with integrity protection

This is description confirmed by the definition of the [//n//]`GH-`[//m//]//rule// extended-BNF pattern defined in §2.1 of [[https://www.ietf.org/rfc/rfc2616.txt|RFC 2616]] as 'a comma-separated list of //rule// with at least //n// and at most //m// items'.

When this reply is parsed by `get_authorization`, request.py only tests for identity with `'auth'`, failing to recognize it as one of the supported modes the server announced, and claims that `"qop 'auth,auth-int' is not supported"`.

* 📜🤖 Added by blurb_it.

* bpo-38686 review fix: remember why.

* fix trailing space in Lib/urllib/request.py

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 14a89c4)

Co-authored-by: PypeBros <PypeBros@users.noreply.github.com>

(cherry picked from commit f8a6316)

Co-authored-by: Sanchit Khurana <54467174+GeniusLearner@users.noreply.github.com>

(cherry picked from commit 386d00c)

Co-authored-by: David Coles <coles.david@gmail.com>

(cherry picked from commit e563a15)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

…is called (GH-17394)

(cherry picked from commit c7c01ab)

Co-authored-by: Steve Dower <steve.dower@python.org>

…H-17366)

Extra newlines are removed at the end of non-shell files. If the file only has newlines after stripping other trailing whitespace, all are removed, as is done by patchcheck.py.
(cherry picked from commit 6bf644e)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

… entries to prevent infinite recursion (GH-17397)

(cherry picked from commit 9bbcbc9)

Co-authored-by: Bruno P. Kinoshita <kinow@users.noreply.github.com>

…et_name__ (GH-17364)

(cherry picked from commit 1bddf89)

Co-authored-by: Florian Dahlitz <f2dahlitz@freenet.de>

SpooledTemporaryFile.rollback() might cause data corruption
when it is in text mode.

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit ea9835c)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>

(cherry picked from commit 02519f7)

Co-authored-by: Tal Einat <taleinat+github@gmail.com>

…hook are invoked (GH-17392)

Also fixes some potential segfaults in unraisable hook handling.

(cherry picked from commit f25875a)

(cherry picked from commit 575d0b4)

Co-authored-by: Ofek Lev <ofekmeister@gmail.com>

(cherry picked from commit fdafa1d)

Co-authored-by: idomic <michael.ido@gmail.com>

* bpo-38449: Add tricky test cases

* bpo-38449: Reflect codereview
(cherry picked from commit 2fe4c48)

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

Make ssl tests less strict and also accept TLSv3 as the default maximum
version. This change unbreaks test_min_max_version on Fedora 32.

https://bugs.python.org/issue38815
(cherry picked from commit 34864d1)

Co-authored-by: torsava <torsava@redhat.com>

…put format (GH-17418)

(cherry picked from commit a62ad47)

Co-authored-by: Matthew Rollings <1211162+stealthcopter@users.noreply.github.com>

Update docstring for `multiprocessing.Pool.map` to mention `pool.starmap()`.

Prev PR: #17367  @aeros

https://bugs.python.org/issue27873
(cherry picked from commit eb48a45)

Co-authored-by: An Long <aisk@users.noreply.github.com>

(cherry picked from commit 808769f)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>

The readline module now detects if Python is linked to libedit at runtime
on all platforms.  Previously, the check was only done on macOS.

If Python is used as a library by a binary linking to libedit, the linker
resolves the rl_initialize symbol required by the readline module against
libedit instead of libreadline, which leads to a segfault.

Take advantage of the existing supporting code to have readline module being
compatible with both situations.
(cherry picked from commit 7105319)

Co-authored-by: serge-sans-paille <serge.guelton@telecom-bretagne.eu>

Use the "volatile" keyword to prevent tail call optimization
on any compiler, rather than relying on compiler specific pragma.
(cherry picked from commit 8b78796)

Co-authored-by: Victor Stinner <vstinner@python.org>

…H-17450)

Since c64a1a6 two assertions were indented and thus ignored when running test_hmac.

This PR fixes it. As the change is quite trivial I didn't add a NEWS entry.


https://bugs.python.org/issue38270
(cherry picked from commit 8943318)


Co-authored-by: stratakis <cstratak@redhat.com>


https://bugs.python.org/issue38270



Automerge-Triggered-By: @tiran

…H-17277)

parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.

https://bugs.python.org/issue38698
(cherry picked from commit bb81549)

Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>

GH-13135)

Break cycle generated when saving an exception in socket.py, codeop.py and dyld.py as they keep alive not only the exception but user objects through the ``__traceback__`` attribute.

https://bugs.python.org/issue36820

Automerge-Triggered-By: @pablogsal
(cherry picked from commit b64334c)

Co-authored-by: Mario Corchero <mcorcherojim@bloomberg.net>

…ges (GH-17480)

Note that the support is not actually enabled yet, and so we won't be publishing these packages. However, for those who want to build it themselves (even by reusing the Azure Pipelines definition), it's now relatively easy to enable.

On most platforms, the `environ` symbol is accessible everywhere.

In a dylib on OSX, it's not easily accessible, you need to find it with
_NSGetEnviron.

The code was caching the *value* of environ. But a setenv() can change the value,
leaving garbage at the old value. Fix: don't cache the value of environ, just
read it every time.
(cherry picked from commit 723f71a)

Co-authored-by: Benoit Hudson <benoit@imgspc.com>

(cherry picked from commit 7ddcd0c)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>

…Python version (GH-17484)

(cherry picked from commit 969ae7a)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>

…ncio functions (GH-16457) (#17496)

https://bugs.python.org/issue37404
(cherry picked from commit 892f9e0)

Co-authored-by: idomic <michael.ido@gmail.com>

test_openssl_version now accepts version 3.0.0.

getpeercert() no longer returns IPv6 addresses with a trailing new line.

Signed-off-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue38820
(cherry picked from commit 2b7de66)


Co-authored-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue38820



Automerge-Triggered-By: @tiran

now contextvars.ContextVar "__class_getitem__" method returns ContextVar class, not None.

https://bugs.python.org/issue38979

Automerge-Triggered-By: @asvetlov
(cherry picked from commit 28c9163)

Co-authored-by: AMIR <31338382+amiremohamadi@users.noreply.github.com>

…H-17503)

This adds a new InvalidMessageID token to the email header parser which can be
used to represent invalid message-id headers in the parse tree.
(cherry picked from commit 68157da)

Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>

…H-17504)

Fix a potential IndexError when passing an empty value to the message-id
parser. Instead, HeaderParseError should be raised.
(cherry picked from commit 3ae4ea1)

Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>

…espace (GH-17421)

https://bugs.python.org/issue38673
(cherry picked from commit 109fc27)

Co-authored-by: Batuhan Taşkaya <47358913+isidentical@users.noreply.github.com>