Another (maybe less invasive and more versatile) approach would be to pass the aforementioned user-data as argument to the enter-method (maybe as enter_with<UserData>(user_data: UserData, …)).

The passed data would be added as another parameter next to the vm, when calling into a function:

let mut my_counter: u32 = 0;
interpreter.enter_with(&mut my_counter, |vm| { … });

fn increment_counter(vm: &VirtualMachine, counter: &mut u32) {
    counter += 1;
}

A huge benefit would be that the user_data doesn't need to be stored anywhere, so that Mutex/RefCell could be avoided.

I don't know how hard that would be to implement in the macro so that there's no ambiguity between ordinary arguments coming from Python and UserData.

Adding a customizable user data field requires customizable area of VirtualMachine, which leads generic type parameter which will be not very useful for most of users.

On the other hand, adding a simple, probably incremental id for each VM can be easier and less fragile.

Typically a pointer value of VM object can be an id of it, but I am not sure it is safe in RustPython.

@coolreader18 Do you have a good idea?

I'm facing a new problem which looks like it would benefit from a solution that doesn't rely on shared state via global variables:

I have an ApplicationState which needs to be updated in a transactional manner. It can be accessed from multiple threads so I wrapped it in a Mutex. While holding the lock I call into an existing python VM which shall perform modification on that state. For this to work, I need to pass the MutexGuard to the callback function which is getting called by the script+vm. I did not find a way to accomplish this task, as the MutexGuard has an attached non-static lifetime - so I cannot store it in a global static variable.

some pseudocode to explain the problem:

fn update(state: &Arc<Mutex<ApplicationState>>) {
    let mut state_lock = state.lock().unwrap();

    // work with the state here

    interpreter.enter(|vm| {
        vm.call_into_some_python_function();
        // this function will in turn call `update_application_state` (see below)
    });    

    // work with the state here even more

    // state will be unlocked here; transaction completed
}

#[pymodule]
pub(crate) mod py_handlers {

    #[pyfunction]
    fn update_application_state(vm: &VirtualMachine) -> PyResult<()> {
        // ← here I need access to the `MutexGuard`
    }

}

If I was able to pass the MutexGuard into the vm-call I might be able to do:

fn update(state: &Arc<Mutex<ApplicationState>>) {
    let mut state_lock = state.lock().unwrap();

    // work with the state here

    interpreter.enter_with_userdata(&mut state_lock, |vm| {
        vm.call_into_some_python_function();
        // this function will in turn call `update_application_state` (see below)
    });    

    // work with the state here even more

    // state will be unlocked here; transaction completed
}

#[pymodule]
pub(crate) mod py_handlers {

    #[pyfunction]
    fn update_application_state(vm: &VirtualMachine, application_state: &mut MutexGuard<ApplicationState>) -> PyResult<()> {
        application_state.update();
    }

}

I could imagine to work around this problem by storing the entire Arc<Mutex<ApplicationState>> in a global variable and add another Mutex-mechanism somewhere to protect the surrounding transaction, but this would be very error-prone, complex and slower.

Is there an easier way to achieve that?

Typically a pointer value of VM object can be an id of it, but I am not sure it is safe in RustPython.

This would imply that a VM never gets moved in memory (in other words Pin IIUC). It's at least a workaround without involving any changes to RustPython - it would be the responsibility of the caller to make this safe.

I tried to implement the idea to get a feeling on how big the fallout would be … doesn't look good :(

Adding the user data to just enter doesn't help, as it needs to get passed into the closure anyway. Then one would have to add user_data to each call that might invoke python code. I tried that for import and had to give up as the generic type of the user data would have to be added to a lot of types, just to pass-through a single value.

I hope there's a way to keep user_data around for the duration of a call into the VM and then (when a rust-function is being called back) use that value as an additional argument. But I'm not sure if this is feasible without major changes.

I'll wait for more feedback now and try to find a hack to work around the Mutex-Problem shown above.

Is it possible to add a parameter to call_into_some_python_function?

Then making a #[pyclass] type including ApplicationState reference (not the & reference, but any kind of) will be possible to expose it as a Python object.

Here is a concept without a new type

fn update(state: &Arc<Mutex<ApplicationState>>) {
    let mut state_lock = state.lock().unwrap();

    // work with the state here
    let ptr = state_lock.deref_mut() as *mut u8;  // This is ugly, but safe as long as this is only used in the lock
    interpreter.enter(|vm| {
        let context = vm.ctx.new_int(ptr);
        vm.call_into_some_python_function(context);
        // this function will pass context to `update_application_state`
    });    

    // work with the state here even more

    // state will be unlocked here; transaction completed
}

call_into_some_python_function was meant as a placeholder to any vm that might trigger a callback. This could be vm.import(), vm.call_*(), etc. where I cannot just add a parameter by myself. (maybe should've expressed that more explicitly, sorry)

Also I guess I would have to access the context by using unsafe Rust, correct?

An idea that's been in my head for a while is an AnyMap stored in the vm that users could insert into - this could also work as a interpreter-distinguishing substitute for the global variables that some cpython modules store things in.