Reinforcement Learning & Quantum Optics: an actor that is not performing as expected in a quantum stage

Question

In the following I would like to pose a problem we have encountered when training neural networks to optimize continuous functions; in particular we have arrived to the toy-