Google DeepMind’s robotics head on general purpose robots, generative AI and office WiFi
Earlier this month, Google’s DeepMind team debuted Open X-Embodiment, a database of robotics functionality created in collaboration with 33 research institutes. The researchers involved compared the system to ImageNet, the landmark database founded in 2009 that is now home to more than 14 million images.
“Just as ImageNet propelled computer vision research, we believe Open X-Embodiment can do the same to advance robotics,” researchers Quan Vuong and Pannag Sanketi noted at the time. “Building a dataset of diverse robot demonstrations is the key step to training a generalist model that can control many different types of robots, follow diverse instructions, perform basic reasoning about complex tasks and generalize effectively.”
At the time of its announcement, Open X-Embodiment contained 500+ skills and 150,000 tasks gathered from 22 robot embodiments. Not quite ImageNet numbers, but it’s a good start. DeepMind then trained its RT-1-X model on the data and used it to train robots in other labs, reporting a 50% success rate compared to the in-house methods the teams had developed.
I’ve probably repeated this dozens of times in these pages, but it truly is an exciting time for robotic learning. I’ve talked to so many teams approaching the problem from different angles with ever-increasing efficacy. The reign of the bespoke robot is far from over, but it certainly feels as though we’re catching glimpses of a world where the general-purpose robot is a distinct possibility.
Simulation will undoubtedly be a big part of the equation, along with AI (including the generative variety). It still feels like some firms have put the horse before the cart here when it comes to building hardware for general tasks, but a few years down the road, who knows?
Vincent Vanhoucke is someone I’ve been trying to pin down for a bit. If I was available, he wasn’t. Ships in the night and all that. Thankfully, we were finally able to make it work toward the end of last week.
Vanhoucke is new to the role of Google DeepMind’s head of robotics, having stepped into the role back in May. He has, however, been kicking around the company for more than 16 years, most recently serving as a distinguished scientist for Google AI Robotics. All told, he may well be the best possible person to talk to about Google’s robotic ambitions and how it got here.
At what point in DeepMind’s history did the robotics team develop?
I was originally not on the DeepMind side of the fence. I was part of Google Research. We recently merged with the DeepMind efforts. So, in some sense, my involvement with DeepMind is extremely recent. But there is a longer history of robotics research happening at Google DeepMind. It started from the increasing view that perception technology was becoming really, really good.
A lot of the computer vision, audio processing, and all that stuff was really turning the corner and becoming almost human level. We starting to ask ourselves, “Okay, assuming that this continues over the next few years, what are the consequences of that?” One of clear consequence was that suddenly having robotics in a real-world environment was going to be a real possibility. Being able to actually evolve and perform tasks in an everyday environment was entirely predicated on having really, really strong perception. I was initially working on general AI and computer vision. I also worked on speech recognition in the past. I saw the writing on the wall and decided to pivot toward using robotics as the next stage of our research.
“Just as ImageNet propelled computer vision research, we believe Open X-Embodiment can do the same to advance robotics,” researchers Quan Vuong and Pannag Sanketi noted at the time. “Building a dataset of diverse robot demonstrations is the key step to training a generalist model that can control many different types of robots, follow diverse instructions, perform basic reasoning about complex tasks and generalize effectively.”
At the time of its announcement, Open X-Embodiment contained 500+ skills and 150,000 tasks gathered from 22 robot embodiments. Not quite ImageNet numbers, but it’s a good start. DeepMind then trained its RT-1-X model on the data and used it to train robots in other labs, reporting a 50% success rate compared to the in-house methods the teams had developed.
I’ve probably repeated this dozens of times in these pages, but it truly is an exciting time for robotic learning. I’ve talked to so many teams approaching the problem from different angles with ever-increasing efficacy. The reign of the bespoke robot is far from over, but it certainly feels as though we’re catching glimpses of a world where the general-purpose robot is a distinct possibility.
Simulation will undoubtedly be a big part of the equation, along with AI (including the generative variety). It still feels like some firms have put the horse before the cart here when it comes to building hardware for general tasks, but a few years down the road, who knows?
Vincent Vanhoucke is someone I’ve been trying to pin down for a bit. If I was available, he wasn’t. Ships in the night and all that. Thankfully, we were finally able to make it work toward the end of last week.
Vanhoucke is new to the role of Google DeepMind’s head of robotics, having stepped into the role back in May. He has, however, been kicking around the company for more than 16 years, most recently serving as a distinguished scientist for Google AI Robotics. All told, he may well be the best possible person to talk to about Google’s robotic ambitions and how it got here.
At what point in DeepMind’s history did the robotics team develop?
I was originally not on the DeepMind side of the fence. I was part of Google Research. We recently merged with the DeepMind efforts. So, in some sense, my involvement with DeepMind is extremely recent. But there is a longer history of robotics research happening at Google DeepMind. It started from the increasing view that perception technology was becoming really, really good.
A lot of the computer vision, audio processing, and all that stuff was really turning the corner and becoming almost human level. We starting to ask ourselves, “Okay, assuming that this continues over the next few years, what are the consequences of that?” One of clear consequence was that suddenly having robotics in a real-world environment was going to be a real possibility. Being able to actually evolve and perform tasks in an everyday environment was entirely predicated on having really, really strong perception. I was initially working on general AI and computer vision. I also worked on speech recognition in the past. I saw the writing on the wall and decided to pivot toward using robotics as the next stage of our research.
Comments
Post a Comment