Hi,
The bug relates to running Reinforcement Learning agents in a synchronous training loop.
When running the training without multiprocessing, all the code functions properly, but when running it with multiprocessing, we get the error shown below in the picture.

Bug report

A clear and concise description of what the bug is.
Include a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example), if possible.
When I run my code without Multiprocessing, everything works properly. I meant to run multiple agents and tried to use ray but found a bug, so I tried multiprocessing, which caused the same bug. This is likely due to ray using multiprocessing under the hood.

A photo of the output is found below

Your environment

https://github.com/openai/gym/tree/master/gym/envs/box2d - CarRacing Environment used, tested for both CarRacing-v1 and v2

• CPython versions tested on: 3.10.4
• Operating system and architecture: Ubuntu 22.04 LTS

Code used for reproducing it: https://pastebin.com/MWiNxcTF
Pip list:
Box2D 2.3.2
box2d-py 2.3.5
ceph 1.0.0
cephfs 2.0.0
cloud-init 22.2
cloudpickle 1.6.0
gym 0.24.1
gym-notices 0.0.7
numpy 1.23.0
Pillow 9.2.0
pip 22.0.2
protobuf 3.20.1
psutil 5.9.1
ptyprocess 0.7.0
pyglet 1.5.0
PyOpenGL 3.1.6
pyOpenSSL 21.0.0
python-apt 2.3.0+ubuntu2
python-debian 0.1.43ubuntu1
pytz 2022.1
PyYAML 5.4.1
scipy 1.8.1
torch 1.12.0
torchfile 0.1.0
virtualenv 20.15.1
visdom 0.1.8.9
wandb 0.12.20