Learning to grasp movable objects based on tactile information

The goal of this industrially sponsored project is to research concepts and methods for tactile-based exploring and grasping of movable objects. Let’s think of a simple example: Imagine the task of grasping an electrical connector (e.g. the one in the left image below) with a robot. The rigid connector is attached to a flexible cable and is movable.

Let’s also assume that the robot limited spatial sensing abilities, and only roughly knows the location of the cable and connector. It is equipped with force and torque sensors as well as with tactile sensors at its finger tips. In order to grasp the connector, the robot first needs to explore the environment, e.g. by touching its surrounding at random locations, and evaluating the tactile information. As soon as the robot obtains evidence to having touched the connector, the exploration should incorporate this accumulated evidence in order to most efficiently estimate the pose of the object.

The project will investigate:

*Object pose estimation with uncertainty of movable objects

The first research subject targets at estimating the object’s pose from a sequence of exploring grasps. Most state-of-the art results assume the object to be static. A scientific challenge of this project is to relax this assumption. This leads to the research aspect of incorporating the object’s movement in relation to the explorative actions. For instance if the robot hand is pushing the object, the object pose might be shifted into the force direction.

*Tactile exploration with incomplete sensor information

The second challenging theme of this project is to deal with incomplete sensor information. While nowadays robots have a number of force and tactile sensors, not all of them are available in each situation. There may also be situations with the robot being in contact with the object, but without having the tactile sensor pads in touch. It is a challenging general question how to organize exploration with such incomplete sensory information.

This project is centered at the intersection of robotics and Machine Learning. In addition to researching interesting scientific concepts, the student will have the chance to work with complex robots and novel sensor systems.

Over the course of the four years, the student will be able to spend 2 months per year at the Honda Research Institute Europe, Germany. During these periods, the student has the unique chance to closely work together with scientists of Honda’s European Fundamental Research unit.