RoboSchool: Machine Learning for Simulated Robotic Locomotion
RoboSchool is a practical experiment into machine learning using Unity and TensorFlow. The application implements reinforcement learning algorithms to train simulated robots to walk by controlling joint rotations. RoboSchool was developed using Unity’s ML Agents, an open-source plugin for Unity that enables games and simulations to serve as environments for training intelligent systems.
The aim of this project was to gain a practical understanding of the principles behind the various forms of machine learning, in particular reinforcement learning and its applications for simulated robotics. The application uses the popular game development engine Unity and their recently released machine learning plugin to train simulated robots to walk, by controlling the rotation of joints. The plugin uses the Proximal Policy Optimisation algorithm and TensorFlow to train a neural network which can be imported directly into Unity for behaviour control. Unity’s Machine Learning Agents plugin is still in a beta form, but is being improved upon and added to rapidly. The application also recreates classic reinforcement learning problems, such as cart-pole, directly in Unity using a C# neural network implementation.
The Proximal Policy Optimisation algorithm shows a lot of promise for locomotion based reinforcements learning tasks. Agents of all varieties learned to learn the motion required to walk towards the goal within 4 - 6 hours of training, which correlates to roughly 800,000 - 1,000,000 update steps per agent. The agents were able to, in general, turn to face the goal if started to their side, but struggled more if it was placed directly behind.