Teaching robots to improvise (using tools)
Humans are very good at using tools in unconventional ways. Don’t have a spoon? You’ll use a pen to stir your coffee. Missing a bolt to hang a lamp? A rubber band will do the trick for now. Need a tray? A book or a tablet will do. This ability to improvise doesn’t come out of nowhere; it stems from cognitive abilities that allow us to make connections between objects, the tools at our disposal, and their uses…
Ganesh Gowrishankar, University of Montpellier
Until now, tool use in robots has been viewed as a problem of exploration and learning: a robot needs to discover how a tool can be used, either by trying out various strategies or by observing and imitating other humans or robots. In our study published in Nature Machine Intelligence, we showed that robots can be taught to think more creatively, “outside the box,” as English speakers would say.
To enable robots to use tools intuitively, just like humans, we first examined how we humans are able to do so.
Toward "tool cognition" in robots
A robot needs to pick up a bucket, but the path is blocked by some obstacles. This robot cannot move the obstacles or jump over them. But it was able to look for—and find—a cleaning stick and use it as a tool to pick up the bucket.
This simple use of a tool may seem obvious to a human, but it is a complex challenge for a robot.
In fact, using tools first requires the robot to understand that it cannot perform a task without a tool (1). It must then find an object in its environment that it can use as a tool to perform the requested task (2). To do so, it must figure out how to use that tool—that is, determine the actions to be performed (3). And finally, of course, it must execute these actions… and thus complete the task (4).
The second and third challenges are fundamental cognitive challenges, ones at which humans seem to excel… while robots are significantly less capable.
Our new "tool cognition" algorithm for robots represents a major step forward in addressing these challenges.
How do humans recognize that an object is suitable for use as a tool for a given task?
In our research, we categorized tools based on how we humans use them. This categorization shows that humans can actually only intuitively recognize a very specific category of tools.
These "Category 1" tools are tools that enable us to perform tasks we are already capable of doing naturally. For example, pliers help us grasp objects, but we can already do that with our fingers; a hammer helps us strike objects, which we can already do with our fist.
Using these “Category 1” tools requires the same motion you would use without a tool: pliers require you to pinch, which is similar to what you would do with your fingers; a hammer requires you to make swinging striking motions, and you would make the same motion without a hammer.
Category 1 tools are probably the most common in human and animal life. The first tools used by humans—stones for striking and cracking open fruits and cutting them—were Category 1 tools, since humans could perform these tasks using similar actions with their fists and fingernails, respectively. And indeed, the tools used by animals can be classified as Category 1.
We refer to tools as “Category 2” tools—tools that enable us to perform tasks that humans can perform without tools, but using very different movements (such as a car jack to lift a car).
Category 3 tools are used to perform tasks that are beyond human capability (such as a vacuum cleaner or a chainsaw).
These Category 2 and 3 tools cannot be used intuitively by humans: users must read instructions, watch another user, or experiment with the tool to figure out how to use it.
Tools that humans use intuitively
In psychology, it is believed that one of the reasons humans are skilled at using tools is that tool use leads to the “embodiment” of tools—that is, the brain eventually comes to view tools as an “extension” of our bodies.
In our research on the “embodiment” of tools and body parts by humans, we have observed that the similarity between an object’s characteristics and the functional characteristics of a body part is important for its embodiment.
This observation, along with our characterization of tools, suggests that humans may use their own limbs as a reference point when identifying Category 1 tools: a hammer is recognized as a tool for striking objects because it closely resembles a fist and an arm; a pair of pliers is recognized as a tool for pinching because it resembles fingers; and similarly, a plate resembles a palm and a bowl resembles cupped hands. By identifying and comparing the posture of our limbs when performing a task, we can identify tools that can be used for the task performed with the same posture.
Using the similarity with its limbs is the key concept used to program our robot.
Teaching robots to recognize tools that look like ones they’re familiar with
Our algorithm uses this concept to enable robots to recognize objects (even those they are seeing for the first time) as tools for each task they are capable of performing “without tools” (that is, using their own limbs, without any additional objects).
Once a robot has learned to perform a task using its limbs, our algorithm allows it to use those limbs as a reference to recognize the tools that help it accomplish the same task. Furthermore, by definition, since Category 1 tools require the same action as when using no tools, the robot can use the same skill (“controller”) to use the tool by simply adjusting its body kinematics to accommodate the new tool.
The robot does not need to have used a single tool before, nor does it need to have observed others using tools.
In our proposal, we also provide a grasping planning algorithm for picking up and using the identified tools. Unlike the algorithms previously described in the scientific literature, ours is the first to enable robots to use tools without first having to learn with those same tools or with other tools.
However, for now, our algorithm only allows robots to use Category 1 tools—that is, tools for tasks that the robot can perform without tools. For Category 2 and 3 tools, learning or observation (followed by imitation) is required, just as with previous tool-use algorithms.
Furthermore, the algorithm can be extended to include tools similar to those already known to the robot: essentially, we replace “similarity to a limb” with “similarity to a known tool,” as suggested by the algorithms described earlier.
But as we have seen, Category 1 tools appear to be the most basic tools used by humans and animals, and we therefore believe that our algorithm represents a significant step toward tool cognition in robots.
Ganesh Gowrishankar, Researcher at the Laboratory of Computer Science, Robotics, and Microelectronics in Montpellier, University of Montpellier
This article is republished from The Conversation under a Creative Commons license. Readthe original article.