Development of an automatic picking system for assistive robots using MR

Welfare vehicles are widely used for disabled or elders, however, it is difficult for users to pick up their package while they are in the vehicle. For these picking tasks, the automatic mobile robot with a robot arm has been used. The pointing of the package and its actions should be instructed at first. For disabled or elders, however, it is desirable to give instructions as simple as possible to operate the robot. Recently, Mixed Reality (MR) devices attract attention to support these tasks. Users will give instructions for the robot intuitively by using an HMD with MR. In this paper, an assist robot in which a robot arm automatically sucks the package is proposed by indicating the rough position using HMD. We have developed a system in which the user commands an assist robot to handle packages from a distance location using an HMD and an assist robot picks packages automatically using AR markers.