Reinforcement Learning is a promising tool to automate controller tuning. However, significant extensions are required for real-world applications to enable fast and robust learning. This work proposes several additions to the state of the art and proves their capability in a series of real world experiments.