Teaching Robots to Improvise (Using Tools)
Humans are very good at using tools in unconventional ways. Don’t have a spoon? You’ll use a pen to stir your coffee. Missing a bolt to hang a lamp? A rubber band will do the trick for now. Need a tray? A book or a tablet will do. This ability to improvise doesn’t come out of nowhere; it stems from cognitive abilities that allow us to make connections between objects, the tools at our disposal, and their uses…
Ganesh Gowrishankar, University of Montpellier
Until now, the use of tools by robots has been viewed as a problem of exploration and learning: a robot needs to discover how a tool can be used, either by trying out various strategies or by observing and imitating other humans or robots. In our study published in *Nature Machine Intelligence*, we showed that robots can be taught to think more creatively—“outside the box,” as English speakers would say.
To enable robots to use tools intuitively, just like humans, we first examined how we, as humans, are able to do so.
Toward "Tool Cognition" in Robots
A robot needs to pick up a bucket, but the path is blocked by some obstacles. This robot cannot move the obstacles or jump over them. But it was able to look for—and find—a cleaning stick and use it as a tool to pick up the bucket.
This simple use of a tool may seem obvious to a human, but it is a complex challenge for a robot.
In fact, using tools first requires the robot to understand that it cannot perform a task without a tool (1). It must then find an object in its environment that it can use as a tool to perform the requested task (2). To do so, it must figure out how to use that tool—that is, determine the actions to take (3). And finally, of course, it must carry out these actions… and thus complete the task (4).
The second and third challenges are fundamental cognitive challenges, ones at which humans seem to excel… while robots are significantly less capable.
Our new "tool cognition" algorithm for robots represents a major step forward in addressing these challenges.
How do humans recognize that an object is suitable for use as a tool for a given task?
In our work, we have categorized tools based on how we, as humans, use them. This categorization shows that humans can, in fact, intuitively recognize only a very specific category of tools.
These “Category 1” tools are tools that enable us to perform tasks that are already part of our natural repertoire. For example, a pair of pliers helps us grasp objects, but we can already do that with our fingers; a hammer helps us strike objects, which we can already do with our fist.
Using these “Category 1” tools requires the same action we would perform without a tool: pliers require you to pinch, which is similar to what you would do with your fingers; a hammer requires you to make swinging striking motions, and you would make the same motion without a hammer.
Category 1 tools are probably the most common in human and animal life. The first tools used by humans—stones for striking and cracking open fruits and for cutting them—were Category 1 tools, because humans could perform these tasks using similar actions with their fists and fingernails, respectively. And in fact, the tools used by animals can be classified as Category 1.
We refer to tools as “Category 2” tools—tools that allow us to perform tasks that humans can perform without tools but using very different movements (such as a car jack to lift a car).
Category 3 tools are used to perform tasks that are beyond human capability (such as a vacuum cleaner or a chainsaw).
These Category 2 and 3 tools cannot be used intuitively by humans: users must read instructions, imitate another user, or explore the tool to figure out how to use it.
Tools that humans use intuitively
In psychology, it is believed that one of the reasons humans are good at using tools is that tool use leads to the “embodiment” of tools—that is, the brain eventually comes to view tools as an “extension” of our bodies.
In our research on the “embodiment” of tools and body parts by humans, we have observed that the similarity between an object’s characteristics and the functional characteristics of a body part is important for its embodiment.
This observation, along with our characterization of tools, suggests that humans may use their limbs as a reference point when identifying Category 1 tools: a hammer is recognized as a tool for striking objects because it closely resembles a fist and an arm; a pair of pliers is recognized as a tool for pinching because it resembles fingers; and similarly, a plate resembles a palm, and a bowl resembles cupped hands. By identifying and comparing the posture of our limbs when we perform a task, we can identify tools that can be used for that task with the same posture.
Using similarity with its limbs is the key concept behind our robot's programming.
Teaching robots to recognize tools that look like ones they're familiar with
Our algorithm uses this idea to enable robots to recognize objects (even those they are seeing for the first time) as tools for each task they are capable of performing “without tools” (that is, using their own limbs, without any additional objects).
Once a robot has learned to perform a task using its limbs, our algorithm allows it to use its limbs as a reference to recognize the tools that help it accomplish the same task. Furthermore, by definition, since Category 1 tools require the same action as when no tools are used, the robot can use the same skill (“controller”) to operate the tool by simply adjusting its body kinematics to accommodate the new tool.
The robot does not need to have ever used a single tool, nor does it need to have observed others using tools.
In our proposal, we also provide a grasping planning algorithm for picking up and using the identified tools. Unlike the algorithms previously described in the scientific literature, ours is the first to enable robots to use tools without first having to learn with those same tools or with other tools.
However, for now, our algorithm only allows robots to use Category 1 tools—that is, tools for tasks that the robot is capable of performing without tools. For Category 2 and 3 tools, learning or observation (followed by imitation) is required, just as with previous tool-use algorithms.
Furthermore, the algorithm can be extended to include tools similar to those already known to the robot: essentially, we replace “similarity to a limb” with “similarity to a known tool,” as suggested by the algorithms described earlier.
But as we have seen, Category 1 tools appear to be the most basic tools used by humans and animals, and we therefore believe that our algorithm represents a significant step toward tool cognition in robots.
Ganesh Gowrishankar, Researcher at the Montpellier Laboratory of Computer Science, Robotics, and Microelectronics, University of Montpellier
This article is republished from The Conversation under a Creative Commons license. Readthe original article.