runtime state and lifecyle

• CPython's runtime relies on a large volume of global state
• some global runtime state is statically initialized, but most is initialized when the runtime is (by Py_Initialize()) or is set lazily on first use
• during runtime finalization (Py_Finalize()) most global runtime state is cleaned up, with resources like memory/files released
• ideally all resources would be released, especially all allocated memory should be freed (but we haven't been rigorous about it)
• i.e. the state of the process after finalization should be as close as possible to what the state was before initialization
• any state that is not cleaned up during finalization effectively leaks
• likewise, any resources not released during finalization effectively leak (e.g. the blocks of memory obmalloc had gotten from the system)
• for normal Python usage (i.e. running the "python3" executable) this is not a problem since such leaks only lasts the short time between finalization and when the process terminates
• it's a different story for embedders

embedders:

• embedders (i.e. very advanced users) are able to execute arbitrary C code (unrelated to the C-API) both before runtime initialization and after finalization
• they are also able to go through the initialize/finalize cycle as many times as they want, again with arbitrary C code in between
• we make a reasonable effort to accommodate such usage (though there are plenty of use cases that need to be ironed out)
• the state that we don't clean up, and resources we don't release, during finalization impact such users--particularly when we leak memory
• pre-3.12:
  • some of that un-finalized runtime state was only initialized during the first runtime initialization (e.g. statically initialized globals) and never re-initialized during subsequent runtime initialization
  • thus, such state from the first init would be used as-is after the subsequent init
  • any un-released resources managed by such state would likewise be used as-is after the subsequent init
• 3.12+:
  • all global runtime state is initialized from scratch during each runtime initialization
  • any state/resources that finalization leaks will not be used after a subsequent init; it will genuinely leak

interpreter lifecycle:

• the runtime also relies on a fair amount of state specific to the execution context, e.g. PyInterpreterState and PyThreadState (which have been around for well over 20 years)
• interpreter init/fini mostly follows the same pattern as the global runtime
• the leak situation applies just as much to interpreters
• however, we're more thorough about interpreter finalization since they are essentially heap-allocated rather than stored in C globals (with, technically, the exception of the main interpreter in 3.12+)
• the main interpreter (often the only interpreter) is a special case since it is initialized during global runtime init, finalized during global runtime fini, and has a few unique jobs

consolidating global runtime state

• over the years, the CPython code base had accrued many thousands of C global variables holding the global runtime state
• in 2014, when I started working on my multi-core Python project (e.g. eventually PEP 684), I realized all that global runtime state was my main obstacle
• having global variables spread throughout the code base made finding a solution challenging
• so in 2017 I created the _PyRuntimeState with the intention of consolidating (pretty much) all the global state there
• consolidation had a number of other benefits that made the effort justifiable outside my multi-core Python project
• in 2017, when I added _PyRuntimeState, I also moved an initial set of key runtime state there
• since then I've been slowly eliminating all the C global variables
• that effort mostly completed in late 2022
• pre-3.12:
  • unconsolidated state was initialized the same as it always had been
  • consolidated state was initialized the same way as it had been, but against a struct field instead of a global variable
• 3.12+:
  • in 2022(?) I introduced a static initializer for _PyRuntime and, on any runtime init after the first one, applied that initializer on _PyRuntime dynamically
  • that made it so that the global runtime state is completely reset during every initialization (any resources leaked by fini would no longer be re-used by a subsequent init)
  • the main interpreter is now embedded in _PyRuntimeState and so is statically allocated and (partially) statically initialized with the rest of _PyRuntime

interpreter isolation

• the solution I settled on in 2014 for my multi-core Python project was to move the GIL to each interpreter (PEP 684)
• doing so required that we isolate interpreters from each other as much as possible
• mostly, this meant following through with consolidating the C global variables and then moving nearly all of them down into each interpreter (PyInterpreterState)
• we started doing so fairly early on (2018?) and were mostly done by 2022, but I saved some key global state for last (e.g. obmalloc, the GIL)
• the situation with leaks in global runtime fini translated down to interpreter fini as we moved the relevant state down (e.g. obmalloc)
• the situation for the main interpreter is mostly the same since its lifecycle is so closely coupled with the global runtime's lifecycle
• for subinterpreters the leaks are strictly the genuine variety since they are dynamically allocated, freed during finalization, and thus never re-used

immortal objects (PEP 683):

• the addition of immortal objects in 3.12 solved some isolation problems for us, particularly relative to the static builtin types
• any object may be made immortal, whether statically allocated or heap-allocated
• lifetime:
  • the lifetime of an "immortal" object isn't necessarily bound to the process; it may be viable only in the current interpreter or the current runtime
  • using an immortal object outside its supported execution context will result in a segfault (access violation) or
  • however, none of that isn't explicitly detailed anywhere nor supported by the C-API
• we've made all interned strings immortal (but haven't been cleaning them up during finalization)

extension modules

global state and interpreter isolation:

• extension modules may store data/objects in C global variables (including indirectly, e.g. an object stored in a dict which is stored in the struct field of a static variable)
• static types actually fall into this category, so the situation is relatively widespread in the community
• the data/objects in those globals often end up shared between interpreters, meaning the extension violates interpreter isolation
• this only matters when subinterpreters are being used (limited currently but potentially much more in the future)
• the addition of heap types (PEP 384) helped with the case of static types, though the transition has been slow (partly due to performance issues that have been worked out only recently)
• the addition of module state (PEP 3121) also helped support per-interpreter isolation, but the transition has likewise been slow (since there wasn't much motivation for people to do it)

single-phase init ("legacy" extensions):

• the extension init function returns the initialized module object
• there is no expectation/requirement that the extension define heap types or use module state or otherwise avoid using C globals
• the init function must take care of the extension's PyModuleDef
• the extension has no opportunity to clean up after itself, neither when the module object is destroyed nor during interpreter/runtime finalization
• we never close the extension module's handle (dlclose(), FreeLibrary())
• trying to import the module in an isolated (per-interpreter GIL) subinterpreter causes an ImportError

multi-phase init:

• multi-phase init (PEP 489, inspired partly by PEP 451) mostly takes us the rest of the way to per-interpreter isolation
• the extension init function returns the module's PyModuleDef, which the import machinery then uses to load the module
• by implementing multi-phase init, the extension promises that it will not store any state/objects in C globals, but will define only heap types and only use module state
• we still never close the extension module's handle (dlclose(), FreeLibrary())

challenges:

• there is no way to tell if an extension is multi-phase init or single-phase init without calling the init function (e.g. there is no distinct "SOABI" tag for a distinct filename)
• the init function can allocate memory, create/store objects, start threads, and register callbacks (incl. in C libraries, e.g. readline)
• multi-phase init functions are not supposed to do any of that but we have no guarantees
• we cannot blindly close an extension module's handle (especially single-phase init) since it might have stored allocated data or an object in a C global, might have started a thread running some of its C code, or might have registered a C callback
• an extension (even multi-phase init) may use a problematic C/etc. library:
  • a library that stores state (incl. callbacks) in global variables, thereby breaking interpreter isolation
  • a library that isn't thread-safe (e.g. relative to temporary state stored in static variables), since there may be races under a per-interpreter GIL
• static types are only partially cleaned up during finalization (e.g. they sometimes(?) hold on to some objects)
• loading the first time only in the main interpreter might mitigate some of the problems

obmalloc state

• we have not (ever?) been freeing the system-allocated pages (stored in the obmalloc state) during runtime finalization
• pre-3.12
  • obmalloc state was initialized statically and runtime finalization did not free the pages
  • all init/fini cycles would share the global obmalloc state and thus system-allocated blocks of memory, so even in embedding scenarios those blocks would mostly not "leak"
  • however, any objects still alive after each fini would never be deallocated, essentially leaking (except where the object was stashed away in a global variable, e.g. inside a static type)
• 3.12+
  • early in the globals consolidation effort (2018?), I moved the obmalloc state from static global variables to _PyRuntimeState, but it caused problems so we reverted the change
  • in late 2022 I finally circled back and moved the obmalloc state to the global runtime state, in _PyRuntimeState.obmalloc (67807cf)
  • at that point, more or less, we started initializing the obmalloc state from scratch at each init--one global obmalloc state is no longer shared across init/fini cycles and all memory pages from each cycle would fully leak
  • in early 2023, not long before the feature freeze, I isolated the obmalloc state to each interpreter, in PyInterpreterState.obmalloc (df3173d)
  • at this point each interpreter was now leaking (after interp fini) all pages it got from the system (the situation for the main interpreter didn't change much since its lifetime is so coupled to the global runtime's)