Controlling robots to do task for human via natural language is one of the most concerned topic in human-robot interaction. In this project, we propose a robot system for executing simple command (pick, place, pour, etc.) on the target object via natural language description. Specifically, given a set of objects (cup, bottle, teddy bear, etc.) on a table and a spoken English description, e.g., ``catch me the bottle on the left'', our robotic system will recognize the audio input, comprehend the instruction, localize target objects, and perform the manipulation task. We name the system SMaRt: Speech-Driven Manipulation Robot.

Demo 1

Demo 2


Demo 3



Techniqual Report Coming Soon...