GH-95736: fix IsolatedAsyncioTestCase to set event loop before calling setup functions #95898

kumaraditya303 · 2022-08-11T18:18:44Z

Issue: IsolatedAsyncioTestCase does not call asyncio.set_event_loop before setUp anymore, asyncio.Runner+PidFdChildWatcher leaves zombie processes #95736

Lib/test/test_unittest/test_async_case.py

gvanrossum · 2022-08-13T17:09:23Z

This is waiting for a resolution of #95900 (comment).

Lib/unittest/async_case.py

…ttest

gvanrossum

Okay this is what I had in mind so LGTM from that perspective, but we need to get a better understanding of what Thomas is asking about the event loop policy.

gvanrossum · 2022-08-16T00:10:06Z

So we've established that in tests deriving from IsolatedAsyncioTestCase, the runner isn't fully initialized yet when setUp is called. (The issue should have that in its topic or at least in the body of the first comment.)

It makes me wonder if the Runner class shouldn't initialize itself eagerly (rather than lazily), but that's way too big a change for 3.11 and there's probably a good reason -- though we've now seen it also causes problems.

@kumaraditya303 Can you shorten the test per my remarks previously in this issue?

I'd also like to invite @serhiy-storchaka to review this, since he did some work on async_case.py just 3 months ago.

Lib/unittest/async_case.py

serhiy-storchaka

In general the solution LGTM, but I have a lot of questions about testing code.

Lib/test/test_unittest/test_async_case.py

Lib/unittest/async_case.py

kumaraditya303 · 2022-08-16T11:02:43Z

Can you shorten the test per my remarks previously in this issue?

Done, FTR the previous test was mostly copied from #95736

serhiy-storchaka

LGTM.

miss-islington · 2022-08-16T15:52:09Z

Thanks @kumaraditya303 for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

…re calling setup functions (pythonGH-95898) (cherry picked from commit 9d51599) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

bedevere-bot · 2022-08-16T15:52:19Z

GH-96027 is a backport of this pull request to the 3.11 branch.

bedevere-bot · 2022-08-16T16:19:22Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-emscripten node (dynamic linking) 3.x has failed when building commit 9d51599.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1056/builds/311) and take a look at the build logs.
Check if the failure is related to this commit (9d51599) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1056/builds/311

Failed tests:

test_os

Summary of the results of the build (if available):

== Tests result: FAILURE ==

338 tests OK.

10 slowest tests:

test_tokenize: 1 min 11 sec
test_unparse: 49.4 sec
test_lib2to3: 44.3 sec
test_capi: 24.7 sec
test_unicodedata: 21.8 sec
test_argparse: 20.1 sec
test_pickle: 17.6 sec
test_statistics: 16.1 sec
test_zipfile: 13.0 sec
test_decimal: 11.6 sec

1 test failed:
test_os

96 tests skipped:
test__xxsubinterpreters test_asyncgen test_asynchat test_asyncio
test_asyncore test_check_c_globals test_clinic test_cmd_line
test_concurrent_futures test_contextlib_async test_ctypes
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_doctest
test_docxmlrpc test_dtrace test_embed test_epoll test_faulthandler
test_fcntl test_file_eintr test_fork1 test_ftplib test_gdb
test_grp test_httplib test_httpservers test_idle test_imaplib
test_interpreters test_ioctl test_kqueue test_launcher test_lzma
test_mmap test_msilib test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_main_handling
test_multiprocessing_spawn test_nis test_openpty test_ossaudiodev
test_pdb test_poll test_poplib test_pty test_pwd test_queue
test_readline test_regrtest test_repl test_select test_selectors
test_smtplib test_smtpnet test_socket test_socketserver test_spwd
test_ssl test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_thread
test_threadedtempfile test_threading test_threading_local test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib2 test_urllib2_localnet test_urllib2net test_urllibnet
test_venv test_wait3 test_wait4 test_webbrowser test_winconsoleio
test_winreg test_winsound test_wsgiref test_xmlrpc test_xmlrpc_net
test_zipfile64 test_zipimport_support test_zoneinfo
0:21:42 load avg: 10.79
0:21:42 load avg: 10.79 Re-running failed tests is not supported with --python host runner option.

Total duration: 21 min 42 sec

Click to see traceback logs

remote: Enumerating objects: 24, done.        
remote: Counting objects:   4% (1/24)        
remote: Counting objects:   8% (2/24)        
remote: Counting objects:  12% (3/24)        
remote: Counting objects:  16% (4/24)        
remote: Counting objects:  20% (5/24)        
remote: Counting objects:  25% (6/24)        
remote: Counting objects:  29% (7/24)        
remote: Counting objects:  33% (8/24)        
remote: Counting objects:  37% (9/24)        
remote: Counting objects:  41% (10/24)        
remote: Counting objects:  45% (11/24)        
remote: Counting objects:  50% (12/24)        
remote: Counting objects:  54% (13/24)        
remote: Counting objects:  58% (14/24)        
remote: Counting objects:  62% (15/24)        
remote: Counting objects:  66% (16/24)        
remote: Counting objects:  70% (17/24)        
remote: Counting objects:  75% (18/24)        
remote: Counting objects:  79% (19/24)        
remote: Counting objects:  83% (20/24)        
remote: Counting objects:  87% (21/24)        
remote: Counting objects:  91% (22/24)        
remote: Counting objects:  95% (23/24)        
remote: Counting objects: 100% (24/24)        
remote: Counting objects: 100% (24/24), done.        
remote: Compressing objects:   8% (1/12)        
remote: Compressing objects:  16% (2/12)        
remote: Compressing objects:  25% (3/12)        
remote: Compressing objects:  33% (4/12)        
remote: Compressing objects:  41% (5/12)        
remote: Compressing objects:  50% (6/12)        
remote: Compressing objects:  58% (7/12)        
remote: Compressing objects:  66% (8/12)        
remote: Compressing objects:  75% (9/12)        
remote: Compressing objects:  83% (10/12)        
remote: Compressing objects:  91% (11/12)        
remote: Compressing objects: 100% (12/12)        
remote: Compressing objects: 100% (12/12), done.        
remote: Total 13 (delta 11), reused 2 (delta 1), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '9d515997f943b7b510268448f372dabcbf957858'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 9d515997f9 GH-95736: fix IsolatedAsyncioTestCase to initialize Runner before calling setup functions  (#95898)
Switched to and reset branch 'main'

../../Objects/obmalloc.c:1450:1: warning: always_inline function might not be inlinable [-Wattributes]
 1450 | arena_map_get(block *p, int create)
      | ^~~~~~~~~~~~~
../../Modules/_testcapi/heaptype.c: In function ‘test_type_from_ephemeral_spec’:
../../Modules/_testcapi/heaptype.c:280:23: warning: unused variable ‘class_ht’ [-Wunused-variable]
  280 |     PyHeapTypeObject *class_ht = (PyHeapTypeObject *)class;
      |                       ^~~~~~~~
../../Modules/_testcapi/heaptype.c:279:19: warning: unused variable ‘class_tp’ [-Wunused-variable]
  279 |     PyTypeObject *class_tp = (PyTypeObject *)class;
      |                   ^~~~~~~~

configure: ../../configure --prefix $(PWD)/target/host --with-pydebug --without-pydebug --with-emscripten-target=node --enable-wasm-dynamic-linking --disable-wasm-pthreads --build=x86_64-pc-linux-gnu --host=wasm32-unknown-emscripten --with-build-python=../build/python
configure: WARNING: using cross tools not prefixed with host triplet
mcc: error: no input files

make: make -j2 all
../../Objects/typeobject.c:74:1: warning: unused function 'static_builtin_index_is_set' [-Wunused-function]
static_builtin_index_is_set(PyTypeObject *self)
^
1 warning generated.
../../Python/initconfig.c:2236:27: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~  ^~~~~~~~~~~~~
../../Python/initconfig.c:128:18: note: format string is defined here
"PYTHONPATH   : '%lc'-separated list of directories prefixed to the\n"
                 ^~~
                 %c
../../Python/initconfig.c:2236:42: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~                 ^~~~~~~~~~~~~
../../Python/initconfig.c:131:58: note: format string is defined here
"PYTHONHOME   : alternate <prefix> directory (or <prefix>%lc<exec_prefix>).\n"
                                                         ^~~
                                                         %c
2 warnings generated.
../../Python/pytime.c:297:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:352:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:518:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
../../Modules/expat/xmlparse.c:3107:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:3106:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:4050:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:4049:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:7681:11: warning: format specifies type 'int' but the argument has type 'ptrdiff_t' (aka 'long') [-Wformat]
          bytesMore, (account == XML_ACCOUNT_DIRECT) ? "DIR" : "EXP",
          ^~~~~~~~~
3 warnings generated.
../../Modules/socketmodule.c:4027:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4080:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4706:54: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
            cmsgh = (i == 0) ? CMSG_FIRSTHDR(&msg) : CMSG_NXTHDR(&msg, cmsgh);
                                                     ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings generated.
../../Modules/_sqlite/connection.c:2125:19: warning: result of comparison of constant 9223372036854775807 with expression of type 'Py_ssize_t' (aka 'long') is always false [-Wtautological-constant-out-of-range-compare]
    if (data->len > 9223372036854775807) {  // (1 << 63) - 1
        ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
1 warning generated.
../../Modules/_testcapi/heaptype.c:279:19: warning: unused variable 'class_tp' [-Wunused-variable]
    PyTypeObject *class_tp = (PyTypeObject *)class;
                  ^
../../Modules/_testcapi/heaptype.c:280:23: warning: unused variable 'class_ht' [-Wunused-variable]
    PyHeapTypeObject *class_ht = (PyHeapTypeObject *)class;
                      ^
2 warnings generated.

make: *** [Makefile:1806: buildbottest] Error 2

Cannot open file '/opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-dl/build/build/build_oot/host/test-results.xml' for upload

bedevere-bot · 2022-08-16T16:23:46Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-emscripten node (pthreads) 3.x has failed when building commit 9d51599.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1050/builds/314) and take a look at the build logs.
Check if the failure is related to this commit (9d51599) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1050/builds/314

Failed tests:

test_os

Summary of the results of the build (if available):

== Tests result: FAILURE ==

342 tests OK.

10 slowest tests:

test_tokenize: 1 min 20 sec
test_unparse: 50.8 sec
test_lib2to3: 49.4 sec
test_capi: 35.1 sec
test_io: 31.0 sec
test_zipfile: 27.5 sec
test_unicodedata: 26.3 sec
test_pickle: 20.0 sec
test_statistics: 16.4 sec
test_decimal: 15.3 sec

1 test failed:
test_os

92 tests skipped:
test__xxsubinterpreters test_asyncgen test_asynchat test_asyncio
test_asyncore test_check_c_globals test_clinic test_cmd_line
test_concurrent_futures test_contextlib_async test_ctypes
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_doctest
test_docxmlrpc test_dtrace test_embed test_epoll test_faulthandler
test_fcntl test_file_eintr test_fork1 test_ftplib test_gdb
test_grp test_httplib test_httpservers test_idle test_imaplib
test_interpreters test_ioctl test_kqueue test_launcher test_lzma
test_mmap test_msilib test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_main_handling
test_multiprocessing_spawn test_nis test_openpty test_ossaudiodev
test_pdb test_poll test_poplib test_pty test_pwd test_readline
test_regrtest test_repl test_select test_selectors test_smtplib
test_smtpnet test_socket test_socketserver test_spwd test_ssl
test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib2 test_urllib2_localnet test_urllib2net test_urllibnet
test_venv test_wait3 test_wait4 test_webbrowser test_winconsoleio
test_winreg test_winsound test_wsgiref test_xmlrpc test_xmlrpc_net
test_xxlimited test_zipfile64 test_zipimport_support test_zoneinfo
0:21:29 load avg: 8.72
0:21:29 load avg: 8.72 Re-running failed tests is not supported with --python host runner option.

Total duration: 21 min 29 sec

Click to see traceback logs

remote: Enumerating objects: 30, done.        
remote: Counting objects:   3% (1/30)        
remote: Counting objects:   6% (2/30)        
remote: Counting objects:  10% (3/30)        
remote: Counting objects:  13% (4/30)        
remote: Counting objects:  16% (5/30)        
remote: Counting objects:  20% (6/30)        
remote: Counting objects:  23% (7/30)        
remote: Counting objects:  26% (8/30)        
remote: Counting objects:  30% (9/30)        
remote: Counting objects:  33% (10/30)        
remote: Counting objects:  36% (11/30)        
remote: Counting objects:  40% (12/30)        
remote: Counting objects:  43% (13/30)        
remote: Counting objects:  46% (14/30)        
remote: Counting objects:  50% (15/30)        
remote: Counting objects:  53% (16/30)        
remote: Counting objects:  56% (17/30)        
remote: Counting objects:  60% (18/30)        
remote: Counting objects:  63% (19/30)        
remote: Counting objects:  66% (20/30)        
remote: Counting objects:  70% (21/30)        
remote: Counting objects:  73% (22/30)        
remote: Counting objects:  76% (23/30)        
remote: Counting objects:  80% (24/30)        
remote: Counting objects:  83% (25/30)        
remote: Counting objects:  86% (26/30)        
remote: Counting objects:  90% (27/30)        
remote: Counting objects:  93% (28/30)        
remote: Counting objects:  96% (29/30)        
remote: Counting objects: 100% (30/30)        
remote: Counting objects: 100% (30/30), done.        
remote: Compressing objects:   6% (1/16)        
remote: Compressing objects:  12% (2/16)        
remote: Compressing objects:  18% (3/16)        
remote: Compressing objects:  25% (4/16)        
remote: Compressing objects:  31% (5/16)        
remote: Compressing objects:  37% (6/16)        
remote: Compressing objects:  43% (7/16)        
remote: Compressing objects:  50% (8/16)        
remote: Compressing objects:  56% (9/16)        
remote: Compressing objects:  62% (10/16)        
remote: Compressing objects:  68% (11/16)        
remote: Compressing objects:  75% (12/16)        
remote: Compressing objects:  81% (13/16)        
remote: Compressing objects:  87% (14/16)        
remote: Compressing objects:  93% (15/16)        
remote: Compressing objects: 100% (16/16)        
remote: Compressing objects: 100% (16/16), done.        
remote: Total 18 (delta 15), reused 3 (delta 2), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '9d515997f943b7b510268448f372dabcbf957858'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 9d515997f9 GH-95736: fix IsolatedAsyncioTestCase to initialize Runner before calling setup functions  (#95898)
Switched to and reset branch 'main'

../../Objects/obmalloc.c:1450:1: warning: always_inline function might not be inlinable [-Wattributes]
 1450 | arena_map_get(block *p, int create)
      | ^~~~~~~~~~~~~
../../Modules/_testcapi/heaptype.c: In function ‘test_type_from_ephemeral_spec’:
../../Modules/_testcapi/heaptype.c:280:23: warning: unused variable ‘class_ht’ [-Wunused-variable]
  280 |     PyHeapTypeObject *class_ht = (PyHeapTypeObject *)class;
      |                       ^~~~~~~~
../../Modules/_testcapi/heaptype.c:279:19: warning: unused variable ‘class_tp’ [-Wunused-variable]
  279 |     PyTypeObject *class_tp = (PyTypeObject *)class;
      |                   ^~~~~~~~

configure: ../../configure --prefix $(PWD)/target/host --with-pydebug --without-pydebug --with-emscripten-target=node --disable-wasm-dynamic-linking --enable-wasm-pthreads --build=x86_64-pc-linux-gnu --host=wasm32-unknown-emscripten --with-build-python=../build/python
configure: WARNING: using cross tools not prefixed with host triplet
mcc: error: no input files

make: make -j2 all
../../Objects/typeobject.c:74:1: warning: unused function 'static_builtin_index_is_set' [-Wunused-function]
static_builtin_index_is_set(PyTypeObject *self)
^
1 warning generated.
../../Python/initconfig.c:2236:27: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~  ^~~~~~~~~~~~~
../../Python/initconfig.c:128:18: note: format string is defined here
"PYTHONPATH   : '%lc'-separated list of directories prefixed to the\n"
                 ^~~
                 %c
../../Python/initconfig.c:2236:42: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~                 ^~~~~~~~~~~~~
../../Python/initconfig.c:131:58: note: format string is defined here
"PYTHONHOME   : alternate <prefix> directory (or <prefix>%lc<exec_prefix>).\n"
                                                         ^~~
                                                         %c
2 warnings generated.
../../Python/pytime.c:297:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:352:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:518:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
../../Modules/expat/xmlparse.c:3107:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:3106:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:4050:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:4049:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:7681:11: warning: format specifies type 'int' but the argument has type 'ptrdiff_t' (aka 'long') [-Wformat]
          bytesMore, (account == XML_ACCOUNT_DIRECT) ? "DIR" : "EXP",
          ^~~~~~~~~
3 warnings generated.
../../Modules/socketmodule.c:4027:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4080:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4706:54: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
            cmsgh = (i == 0) ? CMSG_FIRSTHDR(&msg) : CMSG_NXTHDR(&msg, cmsgh);
                                                     ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings generated.
../../Modules/_sqlite/connection.c:2125:19: warning: result of comparison of constant 9223372036854775807 with expression of type 'Py_ssize_t' (aka 'long') is always false [-Wtautological-constant-out-of-range-compare]
    if (data->len > 9223372036854775807) {  // (1 << 63) - 1
        ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
1 warning generated.
../../Modules/_testcapi/heaptype.c:279:19: warning: unused variable 'class_tp' [-Wunused-variable]
    PyTypeObject *class_tp = (PyTypeObject *)class;
                  ^
../../Modules/_testcapi/heaptype.c:280:23: warning: unused variable 'class_ht' [-Wunused-variable]
    PyHeapTypeObject *class_ht = (PyHeapTypeObject *)class;
                      ^
2 warnings generated.
mcc: warning: USE_PTHREADS + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]

make: *** [Makefile:1805: buildbottest] Error 2

Cannot open file '/opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-pthreads/build/build/build_oot/host/test-results.xml' for upload

bedevere-bot · 2022-08-16T16:25:42Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-wasi 3.x has failed when building commit 9d51599.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1046/builds/391) and take a look at the build logs.
Check if the failure is related to this commit (9d51599) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1046/builds/391

Failed tests:

test_os

Summary of the results of the build (if available):

== Tests result: FAILURE ==

328 tests OK.

10 slowest tests:

test_tokenize: 49.0 sec
test_unparse: 31.3 sec
test_capi: 31.0 sec
test_lib2to3: 30.5 sec
test_unicodedata: 23.8 sec
test_statistics: 17.5 sec
test_pickle: 12.7 sec
test_decimal: 12.4 sec
test_buffer: 12.0 sec
test_set: 8.6 sec

1 test failed:
test_os

106 tests skipped:
test__xxsubinterpreters test_asyncgen test_asynchat test_asyncio
test_asyncore test_bz2 test_check_c_globals test_clinic
test_cmd_line test_concurrent_futures test_contextlib_async
test_ctypes test_curses test_dbm_gnu test_dbm_ndbm test_devpoll
test_doctest test_docxmlrpc test_dtrace test_embed test_epoll
test_faulthandler test_fcntl test_file_eintr test_fork1
test_ftplib test_gdb test_grp test_gzip test_httplib
test_httpservers test_idle test_imaplib test_interpreters
test_ioctl test_kqueue test_launcher test_lzma test_mailbox
test_mmap test_msilib test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_main_handling
test_multiprocessing_spawn test_nis test_openpty test_ossaudiodev
test_pdb test_pipes test_poll test_poplib test_pty test_pwd
test_queue test_readline test_regrtest test_repl test_resource
test_select test_selectors test_smtplib test_smtpnet test_socket
test_socketserver test_spwd test_sqlite3 test_ssl
test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_thread
test_threadedtempfile test_threading test_threading_local test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib test_urllib2 test_urllib2_localnet test_urllib2net
test_urllib_response test_urllibnet test_venv test_wait3
test_wait4 test_webbrowser test_winconsoleio test_winreg
test_winsound test_wsgiref test_xmlrpc test_xmlrpc_net
test_xxlimited test_zipfile64 test_zipimport_support test_zlib
test_zoneinfo
0:04:47 load avg: 3.80
0:04:47 load avg: 3.80 Re-running failed tests is not supported with --python host runner option.

Total duration: 4 min 47 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/Lib/asyncio/base_events.py", line 691, in __del__
    self.close()
  File "/Lib/asyncio/unix_events.py", line 68, in close
    super().close()
  File "/Lib/asyncio/selector_events.py", line 91, in close
    self._close_self_pipe()
  File "/Lib/asyncio/selector_events.py", line 98, in _close_self_pipe
    self._remove_reader(self._ssock.fileno())
                        ^^^^^^^^^^^
AttributeError: '_UnixSelectorEventLoop' object has no attribute '_ssock'


Traceback (most recent call last):
  File "/Lib/test/libregrtest/runtest.py", line 360, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/test/libregrtest/runtest.py", line 317, in _runtest_inner2
    test_runner()
  File "/Lib/test/libregrtest/runtest.py", line 281, in _test_module
    support.run_unittest(tests)
  File "/Lib/test/support/__init__.py", line 1216, in run_unittest
    _run_suite(suite)
  File "/Lib/test/support/__init__.py", line 1090, in _run_suite
    result = runner.run(suite)
             ^^^^^^^^^^^^^^^^^
  File "/Lib/unittest/runner.py", line 208, in run
    test(result)
  File "/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/unittest/suite.py", line 122, in run
    test(result)
  File "/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/unittest/suite.py", line 122, in run
    test(result)
  File "/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/unittest/suite.py", line 122, in run
    test(result)
  File "/Lib/unittest/case.py", line 678, in __call__
    return self.run(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/unittest/async_case.py", line 129, in run
    self._setupAsyncioRunner()
  File "/Lib/unittest/async_case.py", line 122, in _setupAsyncioRunner
    runner.get_loop()
  File "/Lib/asyncio/runners.py", line 82, in get_loop
    self._lazy_init()
  File "/Lib/asyncio/runners.py", line 136, in _lazy_init
    self._loop = events.new_event_loop()
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/asyncio/events.py", line 805, in new_event_loop
    return get_event_loop_policy().new_event_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/asyncio/events.py", line 695, in new_event_loop
    return self._loop_factory()
           ^^^^^^^^^^^^^^^^^^^^
  File "/Lib/asyncio/unix_events.py", line 64, in __init__
    super().__init__(selector)
  File "/Lib/asyncio/selector_events.py", line 56, in __init__
    self._make_self_pipe()
  File "/Lib/asyncio/selector_events.py", line 107, in _make_self_pipe
    self._ssock, self._csock = socket.socketpair()
                               ^^^^^^^^^^^^^^^^^^^
  File "/Lib/socket.py", line 631, in socketpair
    lsock = socket(family, type, proto)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/socket.py", line 233, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 58] Not supported

gvanrossum · 2022-08-16T17:12:25Z

Urm, the WebAssembly buildbot failures are actually due to our test. We need to fix this before we can backport. Apparently by forcing a call to new_event_loop() we are forcing a call to socket.socketpair() which doesn't work on WASM. I don't know what the magic incantation is to skip a test on WASM. I'll ask on Discord.

gvanrossum · 2022-08-16T17:28:04Z

Rest of the discussion in the corresponding issue, starting at #95736 (comment)

…er before calling setup functions (pythonGH-95898) (cherry picked from commit 9d51599) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

…er before calling setup functions (pythonGH-95898) (cherry picked from commit 9d51599) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> Co-authored-by: Serhiy Storchaka storchaka@gmail.com

…er before calling setup functions (pythonGH-95898) (cherry picked from commit 9d51599) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

fix IsolatedAsyncioTestCase to set event loop

f96d467

bedevere-bot added the awaiting review label Aug 11, 2022

📜🤖 Added by blurb_it.

02f369b

kumaraditya303 requested a review from gvanrossum Aug 11, 2022

kumaraditya303 added type-bug An unexpected behavior, bug, or error needs backport to 3.11 expert-asyncio labels Aug 11, 2022

gvanrossum reviewed Aug 13, 2022

View changes

Lib/test/test_unittest/test_async_case.py Outdated Show resolved Hide resolved

kumaraditya303 requested a review from gvanrossum Aug 13, 2022

gvanrossum mentioned this pull request Aug 13, 2022

GH-95899: fix asyncio.Runner to call set_event_loop once #95900

Merged

gvanrossum reviewed Aug 14, 2022

View changes

Lib/unittest/async_case.py Outdated Show resolved Hide resolved

kumaraditya303 added 2 commits Aug 15, 2022

Merge branch 'main' of https://github.com/python/cpython into fix-uni…

bec4f9e

…ttest

add comment & simplify

bf77bb6

kumaraditya303 requested a review from gvanrossum Aug 15, 2022

gvanrossum reviewed Aug 15, 2022

View changes

gvanrossum mentioned this pull request Aug 15, 2022

[3.11] GH-95899: fix asyncio.Runner to call set_event_loop only once (GH-95900) #96003

Merged

gvanrossum requested a review from serhiy-storchaka Aug 16, 2022

graingert reviewed Aug 16, 2022

View changes

Lib/unittest/async_case.py Show resolved Hide resolved

serhiy-storchaka reviewed Aug 16, 2022

View changes

serhiy-storchaka requested a review from asvetlov Aug 16, 2022

graingert mentioned this pull request Aug 16, 2022

IsolatedAsyncioTestCase __del__ could be dangerous #96021

Open

kumaraditya303 added 2 commits Aug 16, 2022

simplify

06cdeea

code review

3324354

kumaraditya303 assigned gvanrossum Aug 16, 2022

serhiy-storchaka approved these changes Aug 16, 2022

View changes

bedevere-bot added awaiting merge and removed awaiting review labels Aug 16, 2022

gvanrossum approved these changes Aug 16, 2022

View changes

gvanrossum merged commit 9d51599 into python:main Aug 16, 2022
14 checks passed

bedevere-bot removed the awaiting merge label Aug 16, 2022

bedevere-bot removed the needs backport to 3.11 label Aug 16, 2022

kumaraditya303 deleted the fix-unittest branch Aug 16, 2022

gvanrossum mentioned this pull request Aug 16, 2022

IsolatedAsyncioTestCase does not call asyncio.set_event_loop before setUp anymore, asyncio.Runner+PidFdChildWatcher leaves zombie processes #95736

Closed

kumaraditya303 mentioned this pull request Aug 17, 2022

[3.11] GH-95736: fix IsolatedAsyncioTestCase to initialize Runner bef… #96042

Merged

encukou mentioned this pull request Aug 17, 2022

gh-95991: Add some infrastructure for testing Limited API in _testcapi #95992

Merged

GH-95736: fix IsolatedAsyncioTestCase to set event loop before calling setup functions #95898

GH-95736: fix IsolatedAsyncioTestCase to set event loop before calling setup functions #95898

kumaraditya303 commented Aug 11, 2022 •

edited by bedevere-bot

gvanrossum commented Aug 13, 2022

gvanrossum left a comment

gvanrossum commented Aug 16, 2022

serhiy-storchaka left a comment

kumaraditya303 commented Aug 16, 2022 •

edited

serhiy-storchaka left a comment

miss-islington commented Aug 16, 2022

bedevere-bot commented Aug 16, 2022

bedevere-bot commented Aug 16, 2022

bedevere-bot commented Aug 16, 2022

bedevere-bot commented Aug 16, 2022

gvanrossum commented Aug 16, 2022

gvanrossum commented Aug 16, 2022

GH-95736: fix IsolatedAsyncioTestCase to set event loop before calling setup functions #95898

GH-95736: fix IsolatedAsyncioTestCase to set event loop before calling setup functions #95898

Conversation

kumaraditya303 commented Aug 11, 2022 • edited by bedevere-bot

gvanrossum commented Aug 13, 2022

gvanrossum left a comment

gvanrossum commented Aug 16, 2022

serhiy-storchaka left a comment

kumaraditya303 commented Aug 16, 2022 • edited

serhiy-storchaka left a comment

miss-islington commented Aug 16, 2022

bedevere-bot commented Aug 16, 2022

bedevere-bot commented Aug 16, 2022

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Aug 16, 2022

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Aug 16, 2022

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

gvanrossum commented Aug 16, 2022

gvanrossum commented Aug 16, 2022

kumaraditya303 commented Aug 11, 2022 •

edited by bedevere-bot

kumaraditya303 commented Aug 16, 2022 •

edited