Improving Grounded Natural Language Understanding through Human-Robot Dialog
Abstract—Natural language understanding for robotics can
require substantial domain- and platform-specific engineering. For example, for mobile robots to pick-and-place objects in an environment to satisfy human commands, we can specify the language humans use to issue such commands, and connect concept words like red can to physical object properties. One way to alleviate this engineering for a new domain is to enable robots in human environments to adapt dynamically— continually learning new language constructions and perceptual concepts. In this work, we present an end-to-end pipeline for translating natural language commands to discrete robot actions, and use clarification dialogs to jointly improve language parsing and concept grounding. We train and evaluate this agent in a virtual setting on Amazon Mechanical Turk, and we transfer the learned agent to a physical robot platform to demonstrate it in the real world.
对机器人技术的自然语言理解可能需要大量针对特定领域和平台的工程。 例如，对于移动机器人在环境中拾取并放置对象以满足人类命令，我们可以指定人类用于发出此类命令的语言，并将诸如红色罐子等概念词连接到物理对象属性。 减轻针对新领域的工程的一种方法是使人类环境中的机器人能够动态适应-不断学习新的语言构造和感知概念。 在这项工作中，我们提出了一个将自然语言命令转换为离散的机器人动作的端到端管道，并使用澄清对话框共同改善了语言解析和概念基础。 我们在Amazon Mechanical Turk上的虚拟环境中训练和评估此代理，然后将学习到的代理转移到物理机器人平台上以在现实世界中进行演示。