Human-Robot Interaction Through Mixed-Initiative Supervised Autonomy

Abstract

Achieving full autonomy in robotic tasks is the ultimate goal for researchers and designers. This goal is motivated by the rapid advancements in automation algorithms and the decreasing cost of robotic hardware. However, full autonomy is still faced by challenges due to unplanned changes and unexpected situations that can occur anytime in the robot’s workspace. Due to various factors, robots might not be able to fully perceive their environments, which increases the possibility of errors and decreases the efficiency of performing the tasks. To overcome such challenges, this thesis proposes to combine the unique and complementary traits of both humans and robots by allowing a human to supervise autonomy tasks and intervene when needed. This intervention is based on the human Situation Awareness (SA) and judgment, or upon requests that are triggered by the robot’s ‘self-confidence’ (SC) in being able to successfully perform the task. In short, the aim of this research is to propose a Mixed-Initiative Supervised Autonomy (MISA) framework and validate its applicability in different robotics applications. To evaluate the applicability and utility of MISA, it is implemented in three different proof of-concept (POC) applications: grid-based collaborative simultaneous localization and mapping (SLAM), automated jigsaw puzzle reconstruction, and autonomous robot grasping. Augmented Reality (AR) (for SLAM) and two-dimensional graphical user interfaces (GUI) (for puzzle re construction and autonomous robot grasping) are custom-designed to enhance human SA and allow intuitive interaction between the human and the agents. The superiority of the MISA framework is demonstrated through experiments. In SLAM, the superior maps produced by MISA preclude the need for post processing of any SLAM stock maps; furthermore, MISA reduces the required mapping time by approximately 50% versus the traditional baseline approach. In automated puzzle reconstruction, the MISA framework out performs both fully autonomous solutions, as well as those resulting from on-demand human intervention prompted by the agent. In robot grasping, applying MISA results in a marked per formance improvement by increasing the end-to-end success rate of the learning-based baseline approach from 35.0% to 87.6% .