The SWIMMER environment is a standard benchmark in reinforcement learning (RL). In particular, it is often used in papers comparing or combining RL methods with direct policy search methods such as genetic algorithms or evolution strategies. A lot of these papers report poor performance on SWIMMER from RL methods and much better performance from direct policy search methods. In this technical report we show that the low performance of RL methods on SWIMMER simply comes from the inadequate tuning of an important hyper-parameter and that, by setting this hyper-parameter to a correct value, the issue can be very easily fixed.

翻译：SWIMMER环境是强化学习的标准基准(RL),特别是在比较或结合RL方法与基因算法或进化战略等直接政策搜索方法的论文中,经常使用RL方法,其中许多文件报告说,RWIMMER方法的绩效不佳,直接政策搜索方法的绩效更好。在本技术报告中,我们表明,SWIMMER方法的绩效较低,只是因为对重要的超参数调得不够,如果将这一超参数设定为正确的值,问题就很容易解决。