Learning to Break Rocks With Deep Reinforcement Learning
Pavan Samtani, Francisco Leiva, Javier Ruíz-del-Solar
Abstract
This work proposes a scheme for learning how to break rocks with an impact hammer. The problem is formulated as a Partially Observable Markov's Decision Process, and then solved through deep reinforcement learning. We propose a simple formulation, requiring only a basic sensorization of the hammer's manipulator, and involving just two discrete actions. We use Dueling Double Deep-Q Networks to parameterize the policy, and wield it with an auxiliary output. The proposed auxiliary task is also trained in simulation, and allows deciding when to stop the operation by detecting the absence of a rock from the observed joints' movement. The resulting policy is tested in a real world experimental environment, using a Bobcat E10 mini-excavator, and various rock types. The results show that a good performance can be obtained in a safe, and robust manner. A video showing some of the obtained results is available in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://youtu.be/tMqDJjK6zPo</uri> .