openai

🐛 Bug

The documentation of DQN agent (https://stable-baselines3.readthedocs.io/en/master/modules/dqn.html) specifies that log_interval parameter is "The number of timesteps before logging". However, when set to 1 (or any other value) the logging is not made at that pace but is instead made every log_interval episode (and not timesteps). In the example below this is made every 200 timesteps.

The following applies to DDPG and TD3, and possibly other models. The following libraries were installed in a virtual environment:

numpy==1.16.4
stable-baselines==2.10.0
gym==0.14.0
tensorflow==1.14.0

Episode rewards do not seem to be updated in model.learn() before callback.on_step(). Depending on which callback.locals variable is used, this means that:

• episode rewards may n

There seem to be some vulnerabilities in our code that might fail easily. I suggest adding more unit tests for the following:

• Custom agents (there's only VPG and PPO on CartPole-v0 as of now. We should preferably add more to cover discrete-offpolicy, continuous-offpolicy and continuous-onpolicy)
• Evaluation for the Bandits and Classical agents
• Testing of convergence of agents as proposed i

Hi, I installed FinRL on my macbook using the codes in the installation block of the MultiCrypto_Trading.ipynb, but while testing I get the following warning: "Length of values (2023) does not match length of index (1999)" and my final episode_return is different from running the same codes on the colab! 0.96 vs 1.08! The issue gets resolved when changing the time_interval to '1d' but for any smal