AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
Laura Smith, Nikita Dhawan, Marvin Zhang, Pieter Abbeel, Sergey Levine
Abstract
Fig. 1: Left: Human instructions for each stage (top) are translated at the pixel level into robot instructions (bottom) via CycleGAN. Right: The robot attempts the task stage-wise, automatically resetting and retrying until the instruction classifier signals success, prompting the human to confirm via key press. Algorithmic details are provided in Algorithm 1.
Topics & Concepts
Computer scienceRobotArtificial intelligenceTask (project management)Process (computing)Reinforcement learningComputer visionHuman–robot interactionConstruct (python library)Robot learningReset (finance)PixelHuman–computer interactionMobile robotEngineeringEconomicsOperating systemProgramming languageSystems engineeringFinancial economicsRobot Manipulation and LearningReinforcement Learning in RoboticsMultimodal Machine Learning Applications