I confirmed that Windows CI fails on this test.

======================================================================
FAIL: test_non_ascii_infile (test.test_json.test_tool.TestTool)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\a\1\s\lib\test\test_json\test_tool.py", line 115, in test_non_ascii_infile
    rc, out, err = assert_python_ok('-m', 'json.tool', infile)
  File "d:\a\1\s\lib\test\support\script_helper.py", line 157, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "d:\a\1\s\lib\test\support\script_helper.py", line 143, in _assert_python
    res.fail(cmd_line)
  File "d:\a\1\s\lib\test\support\script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-X', 'faulthandler', '-I', '-m', 'json.tool', '@test_4856_tmp']

stdout:
---

---

stderr:
---
'charmap' codec can't decode byte 0x81 in position 10: character maps to <undefined>
---

Thanks @methane for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

GH-17464 is a backport of this pull request to the 3.8 branch.

Sorry, @methane, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 808769f3a4cbdc47cf1a5708dd61b1787bb192d4 3.7

GH-17465 is a backport of this pull request to the 3.7 branch.