Incase you feel like starting with RL, just don‘t. That shit does nothing but give you a headache. I just spend 5 hours tweaking hyperparameters, double checking my environment and then I just changed the random seed, and boom! It suddengly converged 😮‍💨🤦‍♂️