Fix test_bdb when running Python is isolated mode.

(cherry picked from commit c0799ec)

…H-10204)

After commit d0f49d2, the output of the
test suite is always buffered as the test output needs to be included in
the JUnit file in same cases (as when a test fails). This has the
consequence that printing or using debuggers (like pdb) in the test
suite does not result in a good user experience anymore.

This commit modifies the test suite runner so it only captures the test
output when the JUnit file is requested to fix the regression so prints
and debuggers are usable again.
(cherry picked from commit 0227748)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

)

The root widget was accessed as a global variable in the Application's method.
(cherry picked from commit a80af77)

Co-authored-by: Daniel Lovell <lovell.daniel92@gmail.com>

…H-10248)

…H-10231). (GH-10254)

(cherry picked from commit 3f819ca)

(cherry picked from commit c9a6168)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

…gument. (GH-10217)

_io.IncrementalNewlineDecoder's initializer possibly assigns out-of-range
value to the bitwise struct field.
(cherry picked from commit b08746b)

Co-authored-by: Xiang Zhang <angwerzx@126.com>

Don't call _Py_FatalError_PrintExc() nor flush_std_files() if the
current thread doesn't hold the GIL, or if the current thread
has no Python state thread.

(cherry picked from commit 3a228ab)

…0268). (GH-10281)

(cherry picked from commit 4b5e62d)

Co-authored-by: Pablo Aguiar <scorphus@gmail.com>

https://bugs.python.org/issue35075
(cherry picked from commit bf46a09)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

* Include memo in the documented signature of copy.deepcopy()

The memo argument is mentioned lower on the doc page under writing a
`__deepcopy__` method, but is not included in the documented function signature.
This makes it easy to miss, and can lead to incorrect/buggy implementations of
`__deepcopy__` -- which is exatly what just happpend to me!
(cherry picked from commit 0200928)

Co-authored-by: Stephan Hoyer <shoyer@gmail.com>

…w match `setup.py` (GH-10289)

This could cause compile errors on macOS or other platforms.
(cherry picked from commit 318ab63)

Co-authored-by: Max Bélanger <aeromax@gmail.com>

Some methods in the os module can accept path-like objects. This is documented in the general documentation but not in the function docstrings. To keep both in sync, the docstrings need to be updated to reflect that path-like objects are also accepted..
(cherry picked from commit b942707)

Co-authored-by: BNMetrics <luna@bnmetrics.com>

Use `__GNUC__` instead of non-existing `__GNUC_MAJOR__`.

(cherry picked from commit e2ed5ad)

(cherry picked from commit fe62d87)

Co-authored-by: İsmail Arılık <arilik.ismail@gmail.com>

GH-10318) (GH-10326)

(cherry picked from commit 59668aa)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>

…ferent lines. (GH-10284) (GH-10335)

Two kind of mistakes:

1. Missed space. After concatenating there is no space between words.

2. Missed comma. Causes unintentional concatenating in a list of strings..
(cherry picked from commit 34fd4c2)

…H-10332) (GH-10338) (GH-10339)

(cherry picked from commit 083a7a1)
(cherry picked from commit 6ad277b)

§31.5.6.3. Importing a source file directly: `module_from_spec` is new in Python 3.5.
(cherry picked from commit 16c8a53)

Co-authored-by: E Kawashima <e-kwsm@users.noreply.github.com>

The section is renamed from "IDLE -- console differences".  It mostly
covers the implications of using custom sys.stdxxx objects.
(cherry picked from commit 5e79090)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

(This should have been done with the first PR for this issue.)
(cherry picked from commit f1d3efc)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

… systems. (GH-10347)

The test depended on '/usr/share/zoneinfo/posixrules' or equivalent
because it set TZ without explicit DST transition rules. At least
on OpenSUSE Tumbleweed that file is linked to '/etc/localtime',
making the test fail with certain local timezones,
such as 'Europe/Moscow' which doesn't have DST transitions since 2011.
(cherry picked from commit f1b9ad3)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>

Add a new subsection to the doc.
(cherry picked from commit 75d9d59)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

A program that runs indefinitely can overfill memory.
(cherry picked from commit 76cd0c3)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

(cherry picked from commit ea6a28c)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

(cherry picked from commit ca03f3b)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

(cherry picked from commit d649910)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

…he index. (GH-10138)

(cherry picked from commit c5eec44)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

(cherry picked from commit ae31e3f)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

(cherry picked from commit bfe1839)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

Allow configure --with-lto to apply to all builds, not just profile-opt builds.

Whether this is actually useful or not must be determined by the person
building CPython using their own toolchain.

My own quick test on x86_64 Debian 9 (gcc 6.3, binutils 2.28) seemed
to suggest that it wasn't, but I expect better toolchains can or will exist
at some point.  The point is to allow it at all.

…H-10400)

The constructor of subprocess.CalledProcessError in the check_output function had an extra None in it.
(cherry picked from commit b939250)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

(cherry picked from commit cdb96f4)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

Pydoc now does not duplicate docstrings for aliases of inherited methods.
(cherry picked from commit a44d34e)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

… (GH-10422)

Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.

Do a bounds check within find_op so it can return before going past the end as a safety measure.

7db3c48#diff-a33329ae6ae0bb295d742f0caf93c137
introduced this off by one error while fixing another one nearby.

This bug was shipped in all Python 3.6 and 3.7 releases.

The included unittest won't fail unless you do a clang msan build.

(cherry picked from commit 49fa4a9)

This typo doesn't affect the result because wrong bits are discarded
on implicit conversion to unsigned char, but it trips UBSan
with -fsanitize=implicit-integer-truncation.

https://bugs.python.org/issue35194
(cherry picked from commit 7a69cf4)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>

1) Convert weird field name "typ" to the more standard "type".
2) For the NUMBER type, convert the value to an int() or float().
3) Simplify ``group(kind)`` to the shorter and faster ``group()`` call.
4) Simplify logic go a single if-elif chain to make this easier to extend.
5) Reorder the tests to match the order the tokens are specified.
   This isn't necessary for correctness but does make the example
   easier to follow.
6) Move the "column" calculation before the if-elif chain so that
   users have the option of using this value in error messages.
(cherry picked from commit b83942c)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>

…0288) (GH-10404)

(cherry picked from commit 556d50d)


Co-authored-by: Julien Palard <julien@palard.fr>

Co-Authored-By: maggyero <gery.ogam@gmail.com>
(cherry picked from commit 009b2f0)

Co-authored-by: Géry Ogam <gery.ogam@gmail.com>

(cherry picked from commit 43a74ab)

Co-authored-by: Srinivas  Thatiparthy (శ్రీనివాస్  తాటిపర్తి) <thatiparthysreenivas@gmail.com>

The System Preferences Dock "prefer tabs always" setting disables some
IDLE features.  Menus are a bit different than as described for Windows
and Linux.
(cherry picked from commit 50ff02b)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

…GH-7252)

https://bugs.python.org/issue33699
(cherry picked from commit b086c8a)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

…_range(). (GH-10442)

This function may access memory which is mapped but is considered
free by libc allocator. It behaves so by design, therefore we
need to suppress sanitizer reports.

GCC doesn't support MSan, so disable only TSan for it.
(cherry picked from commit fd3a91c)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>

The documentation was not covering multiple targets enclosed by
parenthesis nor multiple targets enclosed by brackets, adding them all
would be heavy, an else cover them all and is lighter to read.
(cherry picked from commit 082875d)

Co-authored-by: Julien Palard <julien@palard.fr>

Independently of -> Independent of
(cherry picked from commit cd44980)

Co-authored-by: Srinivas  Thatiparthy (శ్రీనివాస్  తాటిపర్తి) <thatiparthysreenivas@gmail.com>

(cherry picked from commit 9404e77)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

a asynchronous generator -> an asynchronous generator
(cherry picked from commit a9655b7)

Co-authored-by: Windson yang <wiwindson@outlook.com>

…-10493)

Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.

(cherry picked from commit 1584a00)

Contributed by Gregory P. Smith [Google LLC]

Also includes a whitespace fix from make patchcheck to _posixsubprocess.c - unrelated to the main change that makes the CI happy so I'm just doing it now rather than creating a separate PR.

clang's MemorySanitizer understand getc() but does not understand
getc_unlocked().  Workaround: Don't use it on msan builds.
(cherry picked from commit e6c77d8)

Co-authored-by: Gregory P. Smith <greg@krypto.org>

Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
(cherry picked from commit 3015fb8)

Co-authored-by: Gregory P. Smith <greg@krypto.org>

Deprecate _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() in the
documentation. These macros are removed from Python 3.8 by the commit
1a6be91.
(cherry picked from commit 55b4771)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

) (GH-10518)

If tracemalloc is not tracing Python memory allocations,
_PyMem_DumpTraceback() now suggests to enable tracemalloc
to get the traceback where the memory block has been allocated.

Discovered using clang's MemorySanitizer when it ran python3's
test_fstring test_misformed_unicode_character_name.

An msan build will fail by simply executing: ./python -c 'u"\N"'
(cherry picked from commit 746b2d3)

Co-authored-by: Gregory P. Smith <greg@krypto.org>

(cherry picked from commit daeb3c4)

Co-authored-by: l-n-s <supervillain@riseup.net>

(cherry picked from commit bf1355b)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

(cherry picked from commit 8e0b05e)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

(cherry picked from commit 6431347)

Co-authored-by: Julien Palard <julien@palard.fr>



https://bugs.python.org/issue32613

Remove an unnecessary "that":
... will execute that the body ... ->  ... will execute the body ...
(cherry picked from commit 25221b3)

Co-authored-by: Sebastián Ramírez <tiangolo@gmail.com>

"single" needs to be decrefed if PyList_Append() fails.
(cherry picked from commit 4c596d5)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

(cherry picked from commit b65413b)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>

(cherry picked from commit c2ccac7)

Co-authored-by: Andrés Delfino <adelfino@gmail.com>

…0565)

(cherry picked from commit 4edeaea)

Co-authored-by: Srinivas  Thatiparthy (శ్రీనివాస్  తాటిపర్తి) <thatiparthysreenivas@gmail.com>

This missed PyErr_NoMemory() could cause a SystemError when calling
_symtable.symtable().
(cherry picked from commit ad65f15)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

(cherry picked from commit 5a087d5)

Co-authored-by: Srinivas Reddy Thatiparthy (శ్రీనివాస్  రెడ్డి తాటిపర్తి) <thatiparthysreenivas@gmail.com>

… char * as (GH-10580) (GH-10587)

Source of T_STRING: https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Python/structmember.cGH-L51

Source of PyUnicode_FromString
https://github.com/python/cpython/blob/master/Include/unicodeobject.hGH-L702


https://bugs.python.org/issue25438
(cherry picked from commit 689d555)


Co-authored-by: Windson yang <wiwindson@outlook.com>


https://bugs.python.org/issue25438

(cherry picked from commit 8b9c33e)

Co-authored-by: Donald Stufft <donald@stufft.io>

…x. (GH-10330)

(cherry picked from commit 6c48bf2)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

Fixes assertion failures in _datetimemodule.c
introduced in the previous fix (see bpo-31752).

Rather of trying to handle an int subclass as exact int,
let it to use overridden special methods, but check the
result of divmod().
(cherry picked from commit 3ec0f49)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…10619) (GH-10621)

locale.localeconv() now sets temporarily the LC_CTYPE locale to the
LC_MONETARY locale if the two locales are different and monetary
strings are non-ASCII. This temporary change affects other threads.

Changes:

* locale.localeconv() can now set LC_CTYPE to LC_MONETARY to decode
  monetary fields.
* Add LocaleInfo.grouping_buffer: copy localeconv() grouping string
  since it can be replaced anytime if a different thread calls
  localeconv().

(cherry picked from commit 02e6bf7)
(cherry picked from commit 6eff6b8)

…H-10604) (GH-10630)

(cherry picked from commit d936a8f)

Co-authored-by: Julien Palard <julien@palard.fr>

)

I'll watch for 404 on the old URL and will setup an HTTP redirection if needed.
(cherry picked from commit 361e868)

Co-authored-by: Zhiming Wang <github@zmwang.pw>

…anguages. (GH-10558)

(cherry picked from commit 6b73bb5)

Co-authored-by: Julien Palard <julien@palard.fr>

os_read_impl() now also truncates the size to _PY_READ_MAX
on macOS, to avoid to allocate a larger buffer even if _Py_read() is
limited to _PY_READ_MAX bytes (ex: INT_MAX on macOS).
(cherry picked from commit 9a0d7a7)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

…GH-10663)

* bpo-9263: _PyObject_Dump() detects freed memory (GH-10061)

_PyObject_Dump() now uses an heuristic to check if the object memory
has been freed: log "<freed object>" in that case.

The heuristic rely on the debug hooks on Python memory allocators
which fills the memory with DEADBYTE (0xDB) when memory is
deallocated. Use PYTHONMALLOC=debug to always enable these debug
hooks.

(cherry picked from commit 82af0b6)

* bpo-9263: Fix _PyObject_Dump() for freed object (#10661)

If _PyObject_Dump() detects that the object is freed, don't try to
dump it (exit immediately).

Enhance also _PyObject_IsFreed(): it now detects if the pointer
itself looks like freed memory.

(cherry picked from commit 2cf5d32)
(cherry picked from commit 95036ea)

(cherry picked from commit ba57963)

Co-authored-by: takey <taketakeyyy@gmail.com>

…H-4856)

Importing ProcessPoolExecutor may hang or cause an error when the import
accesses urandom on a low resource platform

https://bugs.python.org/issue29877
(cherry picked from commit 1d817e4)

Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

…-10685)

* bpo-35189: Fix eintr_tester.py (GH-10637)

Call setitimer() before each test method, instead of once per test
case, to ensure that signals are sent in each test method.
Previously, only the first method of a testcase class got signals.

Changes:

* Replace setUpClass() with setUp() and replace tearDownClass() with
  tearDown().
* tearDown() now ensures that at least one signal has been sent.
* Replace support.run_unittest() with unittest.main() which has
  a nicer CLI and automatically discover test cases.

(cherry picked from commit aac1f81)

* bpo-35189: Retry fnctl calls on EINTR (GH-10413)

Modify the following fnctl function to retry if interrupted by a signal
(EINTR): flock, lockf, fnctl.

(cherry picked from commit b409ffa)
Co-Authored-By: nierob <nierob@users.noreply.github.com>
(cherry picked from commit 56742f1)

…H-10675) (GH-10688)

The "-I" command line option (run Python in isolated mode) and -X
options (like -X faulthandler) are now also copied by the
multiprocessing and distutils modules when spawning child processes.
Previously, only -E and -s options (enabled by -I) were copied.

subprocess._args_from_interpreter_flags() now copies the -I flag
and options from sys._xoptions like -X dev.

(cherry picked from commit 9de3632)

…GH-10689)

(cherry picked from commit 5b83ef7)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

GH-10592)

(cherry picked from commit 4bb186d)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

(cherry picked from commit 1586958)

Co-authored-by: Boštjan Mejak <bostjan.xperia@gmail.com>

) (GH-10720)

Fix str.format(), float.__format__() and complex.__format__() methods
for non-ASCII decimal point when using the "n" formatter.

Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires
a _PyUnicodeWriter object for the buffer and a Python str object
for digits.

(cherry picked from commit 59423e3)
(cherry picked from commit 6f5fa1b)

… (GH-10725)

(cherry picked from commit 5719f27)

Co-authored-by: Mathieu Dupuy <deronnax@users.noreply.github.com>





https://bugs.python.org/issue35255

Fix mktime() overflow error in test_email: run
test_localtime_daylight_true_dst_true() and
test_localtime_daylight_false_dst_true() with a specific timezone.
(cherry picked from commit cfaafda)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

Copy code from master: add assertions on start and value, replace 'i'
iterator with 'end' pointer for the loop stop condition.

_PyUnicode_FastFill(): fix type of 'data', it must not be constant,
since data is modified by FILL().

…0746) (GH-10749)

Fix also return type for few other functions (clear, releasebuffer).
(cherry picked from commit d4f9cf5)

)

(cherry picked from commit 8152402)
(cherry picked from commit ad8ac54)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

Fix WithThreadsTestPool.test_wrapped_exception()
of test_multiprocessing_fork: join the pool.

WithThreadsTestPool.test_del_pool() is now also decorated
with @support.reap_threads.
(cherry picked from commit b727873)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

…H-10487) (GH-10768)

That section is a tip on how to kill process on Windows for Python prior to 2.7 and 3.2.
3.1 end of support was April 2012 and 2.6 was October 2013, so that hasn't been need for supported versions of Python for more than 5 years. Beside not being needed anymore for a long time, when I read it with the eyes of a Python profane, it makes Python looks bad, like a language from the parts with warts you need to circumvent.
Let's delete that :).
(cherry picked from commit a1c4001)

Co-authored-by: Mathieu Dupuy <deronnax@users.noreply.github.com>

Log TZ to debug a timezone issue... and a few more :-)

(cherry picked from commit 282c03d)

test_eintr no longer fails if the signal handler has not been called.
(cherry picked from commit 2956bff)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

PyType_GetFlags() return type is unsigned long, not long.
(cherry picked from commit 9fbcfc0)

Co-authored-by: Eddie Elizondo <eduardo.elizondorueda@gmail.com>

Fix ResourceWarning in platform.dist() and
platform.linux_distribution() on SuSE and Caldera OpenLinux.

Patch by Ville Skyttä.
(cherry picked from commit 7eeab87)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

* stop using ranlib (closes bpo-31625) (#3815)

Instead, simply pass 's' to ar.

* explicitly list objects for the ar command (#3824)

$^ is not portable.

closes bpo-31625

) (GH-10796)

Fix warnings with gcc 8 for wrapperfunc <-> wrapperfunc_kwds casts.

(cherry picked from commit 1c60715)

* bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099)

Rename also run_tests_slave() to run_tests_worker().

(cherry picked from commit 012f5b9)

* bpo-34279, regrtest: Issue a warning if no tests have been executed (GH-10150)

(cherry picked from commit 9724348)

* test_regrtest: remove unused threading import
(cherry picked from commit 8a73cac)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

(cherry picked from commit a2e3585)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots.

Use a reliable threading.Event to fix these tests.

Other changes:

* Replace send() with sendall()
* Expect specific BlockingIOError rather than generic OSError
* Add a timeout to select() in testAccept() and testRecv()
* Use addCleanup() to close sockets
* Use assertRaises()
(cherry picked from commit ebd5d6d)

Co-authored-by: Victor Stinner <vstinner@redhat.com>

…10822)

Fix an undefined behaviour in the pthread implementation of
PyThread_start_new_thread(): add a function wrapper to always return
NULL.

Add pythread_callback struct and pythread_wrapper() to thread_pthread.h.

(cherry picked from commit 9eea6ea)

…ory (GH-10826) (GH-10832)

Modify asyncio tests to utilize the certificates from the test directory
instead of its own set, as they are the same and with each update they had
to be updated as well.

(cherry picked from commit b062ba7)

get_gmtoff() now returns time_t instead of int to fix the following
Visual Studio warning:

    Modules\timemodule.c(1183): warning C4244: 'return':
    conversion from 'time_t' to 'int', possible loss of data

(cherry picked from commit 503ce5c)

(cherry picked from commit edeca92)

Co-authored-by: Xtreak <tirkarthi@users.noreply.github.com>

). (GH-10825)

(cherry picked from commit b7c2182)

(cherry picked from commit 32bc11c)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…yCArg_repr(). (GH-10853)

Use "ll" instead of the nonstandard "q".
(cherry picked from commit 062cbb6)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>

(cherry picked from commit 4013c17)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

)

Also refactor the call recording implementation and add some notes
about its limitations.
(cherry picked from commit 8ca0fa9)

Co-authored-by: Chris Withers <chris@withers.org>