Anyone who reads just about anything these days will know there are two big things in AI at the moment. They are Generative AI and Humanoid Robots. There is a lot of hype about these two new (to most people) versions of AI and robots, and it has shifted all the major tech companies to have a strategy in one or both of these fields. And together they have made NVIDIA one of the most valuable companies on Earth.
There is a vague notion that these two are related, as certainly the promises made about both of them share an underlying reliance on Deep Learning which first made a splash within the AI community back in 2012, and also a reliance on Reinforcement Learning which has been under development for over 65 years. Deep Learning itself can trace back its intellectual lineage for about 70 years.
But now they have arrived, in the forms that ordinary people see talked about every day, as Generative AI and as Humanoid Robots. This short blog post is to point out some strong similarities between these two “application areas”, similarities in how the hype around them has developed and why that hype may have developed.
This table summarizes the similarities, and small differences, and I discuss them below.
| Generative AI | Humanoid Robots |
|---|---|
| Big tech companies spending $10Bs | Big tech companies spending $10Bs |
| Crazy startup valuations | Crazy startup valuations |
| Era of big learning | Era of big learning |
| Academia priced out (cloud $$) | Academia priced out (human $$) |
| Promise of plentiful low cost white collar labor | Promise of plentiful low cost blue collar labor |
| Lured in by human language | Lured in by human appearance |
| Driven by two AI prediction sins • Performance vs competence • Indistinguishable from magic | Driven by two AI prediction sins • Exponentialism • Speed of deployment |
Hype and Dollars
The first four rows in the table are almost the same for Generative AI and Humanoid Robots.
In both cases there are a massive number of dollars floating about. The big tech companies are all spending tens of billions of dollars on them. All of them have announced their intentions with respect to Generative AI. The announcements on Humanoid Robots have been less, but if you are in robotics, as I am and have been for fifty years, you start to notice which companies are trying to poach your employees or hiring your academic friends.
At the same time there are plenty of startups in Generative AI and Humanoid Robots that have had valuations in the billions of dollars before they have shipped a single product. In some cases the valuations have been in that stratosphere even at the seed stage.
A common element of the two realms is the use of massive amounts of data for machine learning to drive promised products. They are less engineered in a traditional way. Rather they are given lots of data and the black-box machine learning algorithms learn billions of weights in a network and that is where the intelligence of the systems is expected to lie. Interpreting those weights is something that is being researched, but is not well understood. It is less like designing a high performance automobile with lots of engineering, and more like buying a random horse and training it, hoping that it will be well behaved and do what you want it to do. I call this the era of big learning. As with all eras in all technologies this one will eventually be replaced with something else at some point.
For Generative AI the data is digital data that already exists, in the form of all of human writing that is now online, and millions of hours of videos of all sorts of things. There have been plenty of scandals about whether all the big companies have been completely ethical in appropriating copyrighted material into their trained models. This sort of training relies on massive amounts of cloud computation, and in some cases massive amounts of human labor to annotate some of the data. These costs have meant that academics cannot compete with big companies in running their own learning scenarios.
For Humanoid Robots the data has not been extant, but instead has been generated by paying lots of humans to carry out mostly manual tasks, and record data from the humans. Some of it is video data watching the motions of the humans, and some is from gloves on the humans recording finger motions and sometimes forces they are applying when they are tele-operating humanoid robots1. The cost of collecting data from thousands of hours of humans demonstrating manipulation means that academics cannot compete with big companies here either.
The Attractions
The reason we have so much hype about these topics is tied up with my first law of Artificial Intelligence:
When an AI system performs a task, human observers immediately estimate its general competence in areas that seem related. Usually that estimate is wildly overinflated.
The visual appearance of a robot makes a promise about what it can do and how smart it is. It needs to deliver or slightly over deliver on that promise or it will not be accepted.
People interact with a Large Language Model (LLM), generating text on just about any subject they choose. And it generates facile language way better and more human like than any of the previous generations of chatbots that have been developed over the last sixty years. It is the classic con. A fast talker convinces people that there is more to them than there really is. So people think that the LLMs must be able to reason, like a person, must be as knowledgeable as any and all people, and so there for must be able to do any white collar job, as those are the jobs that require a person to be facile with language.
People see a humanoid form robot and its form is an implicit promise that it will ultimately be able to do everything a human can do. If you believe that, with learning, AI is able to do anything as it can learn how to do it from data, then people think that a humanoid robots naturally will be able to do everything a human can do. And then they see it as a cheap form of blue collar labor.
It is the apparent human-ness of these two technologies that both lure people in, and then that promise human level performance everywhere, even when that level has not yet been demonstrated. People think that surely it is just a matter of time.
The Sins People Commit
I have previously written about the seven deadly sins of predicting the future of AI, both on this blog, and in an edited version in Technology Review. I judge that the hype level for both Generative AI and Humanoid Robots are largely driven by two of those seven sins each.
Generative AI
In my analysis above I pointed to Generative AI hype being overestimated because it show very strong performance in using language. This is the AI sin of extrapolating from a narrow performance to believing there must be much more general competence. The problem is that any person who has facile language performance is usually quite competent in being able to reason, to know the truth and falsity of many propositions, etc. But LLMs do not have any of these, rather they have only the ability to predict likely next words that ought to follow a string of existing words. Academics, VCs, startup founders, and many others though have a strong belief there must be an emergent system within the learned weights that is able to reason, judge, estimate, etc. Many of them are betting with papers they write, cash they invest, or sweat equity, that this really must be true. Perhaps we have a bit too much of Narcissus in us.
The second sin that leads to overhype is the “indistinguishable from magic” sin. Arthur C. Clarke said that “any sufficiently advanced technology is indistinguishable from magic”. He meant that if technology is very much advanced from what you are used to you no longer have a mental model of that technology can and cannot do and so can’t know its limitations. Again, this is what happens with generative AI, as it can perform amazingly well, and so people do not understand its limitations, partly because they keep forgetting how it works, enthralled instead by the spectacular results in generating great language.
Humanoid Robots
The arguments for humanoid robots are based on the idea that they will be able to do everything that a human can do.
But there is an additional argument that many people make. They argue that just as computation has gotten exponentially cheaper over time, so too will humanoid robots. But this is the fallacy of exponentialism. Computation got cheaper exponentially as we were able to reduce the amount of material in a circuit and still have it work. And the key to that being true is the digital abstraction of detecting whether a there is a current flowing or not, and treating that as a digital 1 or 0. When digital electronics started out in silicon there were massive numbers of electrons flowing in the currents used. We were able to halve that about twenty two times, so reducing it by a factor of 4 million and the systems still worked. But you can’t do that with mechanical systems that are doing real work with real payloads. Yes perhaps, just perhaps, today’s physical systems are about four times as large as they need to be to lift and move the objects that a human lifts and moves. Ultimately then we might get a reduction of four times in price. But that leads to nothing like the pennies per hour that exponentialists predict for humanoid robot labor costs.
The other sin is belief that a lab demonstration can result in deployment at scale in just a year or two. I first took a ride in a self driving Waymo car on a freeway in Mountain View California in 2012. (And that was 22 years after Ernst Dickmanns had first had his autonomous vehicle driving on the autobahns outside of Munich.) Thirteen years after my first ride I can still only take a ride in a Waymo, or any other autonomous vehicle, in a handful of US cities, and all of them in limited geofenced areas. It takes decades to deploy physical systems at scale, even when they don’t need changes in infrastructure.
Conclusion
Generative AI and Humanoid Robots tap into the fantasy of infinite wealth from new technologies. Many new technologies have undoubtedly made the lives of humans better, and both these technologies may well do so. But it will not be at the physical scale or short timescale than proponents imagine. People will come to regret how much capital they have spent in these pursuits, both at existing companies and in start ups.
Footnote
1. I fear that this will not actually work in practice. The key to dexterous manipulation is what happens when our fingers and hands are in contact with objects, and how we respond to the forces and touch data we receive. We now know that there are at least 18 families of neurons involved in our sense of touch and it is much more complex and active than our consciousness tells us. No-one is collecting rich touch data nor how the human responds to the detailed forces felt and applied when in contact with objects. Look at the many much heralded videos of humanoid robots grasping objects. They demonstrate just a first draft cartoon manipulation capability.