Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose _Py_NewInterpreter() as Py_NewInterpreterFromConfig() #98608

Closed
ericsnowcurrently opened this issue Oct 24, 2022 · 0 comments
Closed

Expose _Py_NewInterpreter() as Py_NewInterpreterFromConfig() #98608

ericsnowcurrently opened this issue Oct 24, 2022 · 0 comments
Assignees
Labels
3.12 expert-subinterpreters interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@ericsnowcurrently
Copy link
Member

ericsnowcurrently commented Oct 24, 2022

A while back I added _Py_NewInterpreter() (a "private" API) to support configuring the new interpreter. Ultimately, I'd like to adjust
the signature a little and then make the function part of the public API (as Py_NewInterpreterFromConfig()).

My plan:

  1. change the argument to a new _PyInterpreterConfig struct
  2. rename the function to Py_NewInterpreterFromConfig(), inspired by Py_InitializeFromConfig() (takes a PyInterpreterConfig instead of isolated_subinterpreter)
  3. split up the boolean isolated_subinterpreter into the corresponding multiple granular settings
    • allow_fork
    • allow_subprocess
    • allow_threads
  4. drop PyConfig._isolated_interpreter

Note that the current default (Py_NewInterpeter() and Py_Initialize*()) allows fork, subprocess, and threads, and the optional "isolated" interpreter disables all three. I'm not planning on changing any of that here.

My main objective here is to expose the existing API in a way that we can do the following afterward:

  • stop giving the option to disallow subprocess (i.e. drop PyInterpreterConfig.allow_subprocess)
  • add an option to disallow just "exec" instead
  • stop disallowing threads as a default behavior for an "isolated" interpreter (we'd still keep the option though)
  • add the option to disallow daemon threads
  • add an option to check if each extension supports running in multiple interpreters
  • add other options for PEP 684 (per-interpreter GIL)
@ericsnowcurrently ericsnowcurrently added type-feature A feature request or enhancement interpreter-core (Objects, Python, Grammar, and Parser dirs) expert-subinterpreters 3.12 labels Oct 24, 2022
@ericsnowcurrently ericsnowcurrently self-assigned this Oct 25, 2022
ericsnowcurrently added a commit that referenced this issue Oct 26, 2022
…() (gh-98609)

(see #98608)

This change does the following:

1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig`  instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
   * allow_fork
   * allow_subprocess
   * allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`

The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three.  None of that changes here; the defaults are preserved.

Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved.  This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`.  I'd rather just avoid that mess from the start for `_PyInterpreterConfig`.  We can preserve it later if we find an actual need.

This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).

(Note that this PR adds "private" symbols.  We'll probably make them public, and add docs, in a separate change.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 expert-subinterpreters interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
Projects
Status: Done
Development

No branches or pull requests

1 participant