Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-Loading the sys or builtins Modules is Problematic #102660

Open
ericsnowcurrently opened this issue Mar 13, 2023 · 1 comment
Open

Re-Loading the sys or builtins Modules is Problematic #102660

ericsnowcurrently opened this issue Mar 13, 2023 · 1 comment
Labels
3.12 expert-subinterpreters interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Mar 13, 2023

The sys and builtins modules are special in that they are initialized early in interpreter startup, mostly before the import system. They aren't initialized like other modules. Furthermore, the data they expose is fundamental to the interpreter. Reloading the modules doesn't make much sense.

The simplest thing to do is to disallow manually loading them (e.g. via the loader). At the very least we should handle m_copy for these modules carefully. (This came up while working on gh-101660.)

Linked PRs

@ericsnowcurrently ericsnowcurrently added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) expert-subinterpreters 3.12 labels Mar 13, 2023
@ericsnowcurrently ericsnowcurrently self-assigned this Mar 13, 2023
ericsnowcurrently added a commit that referenced this issue Mar 14, 2023
…h-102661)

It doesn't make sense to use multi-phase init for these modules. Using a per-interpreter "m_copy" (instead of PyModuleDef.m_base.m_copy) makes this work okay. (This came up while working on gh-101660.)

Note that we might instead end up disallowing re-load for sys/builtins since they are so special.

#102660
@ericsnowcurrently
Copy link
Member Author

I'm leaving this open since we haven't settled whether or not we should disallow directly loading sys/builtins. (That isn't critical to settle in the short-term, so I'm not going to focus much on it for now.)

@ericsnowcurrently ericsnowcurrently removed their assignment Mar 14, 2023
carljm added a commit to carljm/cpython that referenced this issue Mar 14, 2023
* main: (50 commits)
  pythongh-102674: Remove _specialization_stats from Lib/opcode.py (python#102685)
  pythongh-102660: Handle m_copy Specially for the sys and builtins Modules (pythongh-102661)
  pythongh-102354: change python3 to python in docs examples (python#102696)
  pythongh-81057: Add a CI Check for New Unsupported C Global Variables (pythongh-102506)
  pythonGH-94851: check unicode consistency of static strings in debug mode (python#102684)
  pythongh-100315: clarification to `__slots__` docs. (python#102621)
  pythonGH-100227: cleanup initialization of global interned dict (python#102682)
  doc: Remove a duplicate 'versionchanged' in library/asyncio-task (pythongh-102677)
  pythongh-102013: Add PyUnstable_GC_VisitObjects (python#102014)
  pythonGH-102670: Use sumprod() to simplify, speed up, and improve accuracy of statistics functions (pythonGH-102649)
  pythongh-102627: Replace address pointing toward malicious web page (python#102630)
  pythongh-98831: Use DECREF_INPUTS() more (python#102409)
  pythongh-101659: Avoid Allocation for Shared Exceptions in the _xxsubinterpreters Module (pythongh-102659)
  pythongh-101524: Fix the ChannelID tp_name (pythongh-102655)
  pythongh-102069: Fix `__weakref__` descriptor generation for custom dataclasses (python#102075)
  pythongh-98169 dataclasses.astuple support DefaultDict (python#98170)
  pythongh-102650: Remove duplicate include directives from multiple source files (python#102651)
  pythonGH-100987: Don't cache references to the names and consts array in `_PyEval_EvalFrameDefault`. (python#102640)
  pythongh-87092: refactor assemble() to a number of separate functions, which do not need the compiler struct (python#102562)
  pythongh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (python#102631)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 expert-subinterpreters interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
Projects
Status: No status
Development

No branches or pull requests

1 participant