Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pluggable optimizer API #104584

Open
markshannon opened this issue May 17, 2023 · 3 comments
Open

Pluggable optimizer API #104584

markshannon opened this issue May 17, 2023 · 3 comments
Labels
3.13 new features, bugs and security fixes performance Performance or resource usage

Comments

@markshannon
Copy link
Member

markshannon commented May 17, 2023

We need an API for optimizers to be plugged in to CPython.

The proposed model is that of client server, where the VM is the client and the optimizer is the server.
The optimizer registers with the VM, then VM calls the optimizer when hotspots are detected.

The API:

type struct {
    OBJECT_HEADER;
    _PyInterpreterFrame *(*execute)(PyExecutorObject *self, _PyInterpreterFrame *frame, PyObject **stack_pointer);
    /* Data needed by the executor goes here, but is opaque to the VM */
} PyExecutorObject;

/* This would be nicer as an enum, but C doesn't define the size of enums */
#define PY_OPTIMIZE_FUNCTION_ENTRY 1
#define PY_OPTIMIZE_RESUME_AFTER_YIELD 2
#define PY_OPTIMIZE_BACK_EDGE 4
typedef uint32_t PyOptimizerCapabilities;

type struct {
    OBJECT_HEADER;
    PyExecutorObject *(*compile)(PyOptimizerObject* self, PyCodeObject *code, int offset);
    PyOptimizerCapabilities capabilities;
    float optimization_cost;
    float run_cost;
    /* Data needed by the compiler goes here, but is opaque to the VM */
} PyOptimizerObject;

void _Py_Executor_Replace(PyCodeObject *code, int offset, PyExecutorObject *executor);

int _Py_Optimizer_Register(PyOptimizerObject* optimizer);

The semantics of a PyExecutorObject is that upon return from its execute function, the VM state will have advanced N instructions. Where N is a non-negative integer.

Full discussion here: faster-cpython/ideas#380

This is not a replacement for PEP 523. That will need a PEP. We should get this working first, before we consider replacing PEP 523.

Linked PRs

@markshannon markshannon added performance Performance or resource usage 3.13 new features, bugs and security fixes labels May 17, 2023
@markshannon
Copy link
Member Author

Note that the above API is just the initial version to support our work on speeding up Python 3.13.
It will probably need to extended to support PyTorch Dynamo and other users of PEP 523 that cannot use PEP 669, but that is for another issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 new features, bugs and security fixes performance Performance or resource usage
Projects
None yet
Development

No branches or pull requests

2 participants
@markshannon and others