gh-83076: 3.8x speed improvement in (Async)Mock instantiation #100252
Conversation
|
| Name | Link |
|---|---|
| 246772c | |
| https://app.netlify.com/sites/python-cpython-preview/deploys/639a0b711b176f0007bd9ca0 |
carljm
changed the title
Summary: Modified port of D35118477 (793f7d0) from 3.8, and backport of upstream PR python/cpython#100252 Changes from the 3.8 version: - We don't need to keep `_spec_asyncs` around at all, it's not public API. - Changing the `__code__` attribute of an `AsyncMock` to be a real code object instead of a mock is probably fine, but it's technically backwards incompatible and could break someone's test suite, so I wasn't sure upstream would go for it. Instead we can create the mock object in a more manual way that avoids a lot of the introspection cost (by doing it just once up front.) Reviewed By: mpage Differential Revision: D42039568 fbshipit-source-id: a939f9c7b0c2158ea4d804fdf5d506826531a510
* main: Improve stats presentation for calls. (pythonGH-100274) Better stats for `LOAD_ATTR` and `STORE_ATTR` (pythonGH-100295) pythongh-81057: Move the Cached Parser Dummy Name to _PyRuntimeState (python#100277) Document that zipfile's pwd parameter is a `bytes` object (python#100209) pythongh-99767: mark `PyTypeObject.tp_watched` as internal use only in table (python#100271) Fix typo in introduction.rst (python#100266) pythongh-78997: AttributeError if loading fails in LibraryLoader.__getattr__ pythonGH-100234: Set a default value for random.expovariate() (pythonGH-100235) Remove uninformative itertools recipe (pythonGH-100253) pythonGH-99767: update PyTypeObject docs for type watchers (pythonGH-99928) Move stats for the method cache into the `Py_STAT` machinery (pythonGH-100255) pythonGH-100222: fix typo _py_set_opocde -> _py_set_opcode (pythonGH-100259) pythonGH-100000: Cleanup and polish various watchers code (pythonGH-99998) pythongh-90111: Minor Cleanup for Runtime-Global Objects (pythongh-100254)
* main: pythongh-99540: Constant hash for _PyNone_Type to aid reproducibility (pythonGH-99541) pythongh-100039: enhance __signature__ to work with str and callables (pythonGH-100168) pythongh-99830: asyncio: Document returns of remove_{reader,writer} (python#100302) "Compound statement" docs: Fix with-statement step indexing (python#100286) pythonGH-90043: Handle NaNs in COMPARE_OP_FLOAT_JUMP (pythonGH-100278)
|
Since this is a performance improvement I would suggest keeping them in main branch to target 3.12 release to get some more testing during alpha and beta candidates for regression than backporting it to 3.11 that had a patch release and this is not a bug fix. |
|
@tirkarthi - if there was a serious performance regression, then I'd suggest it's worth backporting. |
|
The performance regression here occurred from 3.7 to 3.8. At least for basic instantiation of a I don't have strong feelings about backport. For work where we are running into this regression, we use a patched Python anyway, we already carried a patch for this in 3.8 and I will backport this to whatever pre-3.12 versions we need it in, so I don't need it backported upstream. I don't know what the usual policy in Python is for backporting perf regression fixes. The fact that this regressed in 3.8 and didn't get fixed until 3.12 suggests it's not a big issue for many people (which isn't surprising, I don't think many people have test suites large enough where a regression in Mock creation time would matter.) So maybe that suggests there's no pressing need to backport? OTOH I think the changes are safe and it's a nice speed boost. |
* main: pythongh-89727: Fix os.walk RecursionError on deep trees (python#99803) Docs: Don't upload CI artifacts (python#100330) pythongh-94912: Added marker for non-standard coroutine function detection (python#99247) Correct CVE-2020-10735 documentation (python#100306) pythongh-100272: Fix JSON serialization of OrderedDict (pythonGH-100273) pythongh-93649: Split tracemalloc tests from _testcapimodule.c (python#99551) Docs: Use `PY_VERSION_HEX` for version comparison (python#100179) pythongh-97909: Fix markup for `PyMethodDef` members (python#100089) pythongh-99240: Reset pointer to NULL when the pointed memory is freed in argument parsing (python#99890) pythongh-99240: Reset pointer to NULL when the pointed memory is freed in argument parsing (python#99890) pythonGH-98831: Add DECREF_INPUTS(), expanding to DECREF() each stack input (python#100205) pythongh-78707: deprecate passing >1 argument to `PurePath.[is_]relative_to()` (pythonGH-94469)
This PR makes several performance optimizations to instantiation of
Mock()andAsyncMock(). The changes are discussed in more detail in the linked issue #83076. The changes result in a net speedup of 3.8x for creation of both kinds of mocks.Before this PR:
After this PR:
There's also one simple code cleanup here: the
AsyncMagicMixin.__init__method was an exact duplicate of the inheritedMagicMixin.__init__. This is unnecessary duplication, we can just inherit it instead.