Learning robotic motor skills for dynamic grasping, catching, and dexterous in-hand manipulation

Abstract

Endowing robots with human-level grasping and manipulation skills is an appealing yet challenging research topic over decades. Towards more extensive functionalities, the robots should interact with the objects and environment in a more robust, efficient and intelligent way. With the rapid development of deep learning, merging the learning-based algorithms into the perception and control loop of the robot exhibits promising performances. In this thesis, we explore the implementation of model-free deep reinforcement learning (DRL) and learning from demonstrations (LfD) in acquisition of dynamic grasping and manipulation motor skills of a robotic hand-arm system.

We propose a systematic framework for end-to-end learning robotic control policies with model-free DRL, including simulation set-up, reward design and sim-to-real transfer. The DRL-based training framework is evaluated by three case studies with different robotic tasks and research emphases. We first introduce the framework in Chapter 3, with a focus on the design of reward function and special initial training states. The trained policy coordinately controls the robotic hand and arm for dynamic reaching, grasping and re-grasping of objects sliding on the ground. We further propose a multi-modular structure consisting of three control policies trained with proposed framework (Chapter 5). With seamless cross-module integration achieved by the gating policy network, the robot can catch in-flight objects with mitigated impact forces. In Chapter 6, dexterous in-hand manipulation skills with tactile feedback is trained from scratch in simulation and directly transferred to real robot. We focus on the exploitation of tactile perception and sim-to-real transfer methods.

Moreover, in Chapter 4 we propose a low-cost method to collect human demonstration data for supervised learning. Using only proprioceptive sensing, the trained neural network based controllers can grasp property-unknown objects with adaptive grasping forces.