This post is not about research or developing software for robots. Instead it is some tips on how to go about building robots for mass deployment and how to leverage those deployed robots for improving your product.

The four tips are straightforward but I explain them more below.

1. Use other people’s supply chain scale wherever possible.
2. Changing infrastructure up front eases robot success but kills ROI for customers and their ease of buying decisions.
3. New infrastructure already developed for people is often good infrastructure for robots.
4. Deployed robots can collect real world data.

1. Other People’s Supply Chain Scale

When you start deploying a robot it is unlikely to be initially in large numbers unless it is a very low price.  When we started selling Roombas at iRobot our first manufacturing batch was 70,000 robots, which is enormous by most robot deployment standards. We got to millions of robots per year. But these numbers are unusual for robots. Even so we got supply chain economies of scale at the millions we did not have at the tens of thousands.

How are you going to get economies of scale when you are shipping just a couple of robots per month, or even tens per week? And really how big an influence can you have over suppliers even when the volume is enormous, for you, at a small number of hundreds of robots per week?

My advice is to find someone else’s big juicy supply chain and grab a hold of that for yourself. Juicy in this context means three things. (1) Supply chains with significant downward price pressures. (2) Supply chains with enormous volumes. (3) Supply chains with many different suppliers, supplying standard parts, and all competing, and operating, in multiple geographies.

For example, you can use exactly the same motors that someone else is using in the millions per year, preferably a standard, and preferably ones that are built by multiple suppliers. Then buy those same motors from one or more of those standard suppliers, though first make sure they have adequate quality control and and quality guarantees. Ten years ago it made sense to get custom windings for motors in Eastern Europe. Now you should try to use existing motors from large scale consumer products.

Do the same for battery packs, or at least battery cells. Fifteen years ago it made sense to have unique battery chemistries both for robot performance and for the regulatory requirements at the geographical point of deployment. Now there are so many at scale different battery cells and chemistries available for at scale consumer products (including electric vehicles) it makes sense to pick the closest fit and use those in your robot.

Do the same for every other part of your robot that you can.

Doesn’t this make designing the production robot harder, getting standard parts rather than ones optimized for your robot?  Yes, and no.  It makes design a slightly different process, but one that rewards you handsomely in terms of lower BOM (Bill Of Materials) cost, and in stability of supply. Those of us (which is just about everyone) who went through the turbulence of COVID-era supply chains can testify to how that turbulence can kill the stability of your careful designs.

Oh, and by the way, you are already, to at least a limited extent, practicing this type of design. For your fasteners no manufacturing engineer is going to let you get away with deciding on an M5.3 bolt [an M5 bolt is 5mm, and an M6 bolt is 6mm, and they are standard sizes, with no standard in between], with a custom head, and a custom pitch, and a custom length of 10.27mm. No, they are going to insist that you use an M5 or an M6 bolt, with one of the many standard heads and pitches and one of the many standard lengths.  And the reasons for that are precisely the same as the reasons I gave above for motors and battery cells, and processor boards, connectors, etc. There is an enormous supply chain for standard sized bolts, so they are way cheaper than custom sized bolts, by two or three orders of magnitude.

2. Changing Infrastructure

I have a rule when designing robots that are going to be deployed to real customers. No one in the company is allowed to say “the customer/end-user can just …”. If we are asking a customer or end-user to do something they wouldn’t naturally do already we are making it harder for them to use our product, compared to what they are already doing, or making it more expensive for them to install our product. This is true of both user interfaces and infrastructure.

As a business you need to decide whether you are selling infrastructure, or selling something that goes into an existing environment. A robot that can move about is a powerful thing to add to an existing environment because it provides mobility and mobile sensors without, in general, having to change the infrastructure.

The Roomba was an easy sale, not just for its price, but because it did not demand that people had to install some infrastructure in their home environment before they could have a Roomba clean their floors. All that a Roomba needed was a standard electrical outlet into which a human could plug its charger. Nothing else.

Since we have been working at Robust AI on a new class of autonomous mobile robots for warehouses and factories, many people (mostly outside of the company, and not potential customers) have suggested all sorts of infrastructure changes that would make it easier to deploy our robots. These have included rubber floors, floors with a wireless recharging system underneath them so the robots can stay charged at all times, radio or laser beacons for localization of the robot, QR codes on every pillar, also for localization, structured lighting systems for the whole interior of the building, separate floors for human driven forklifts and the robots, etc., etc.

All these things would be additional expense for deploying the robots. It would have three bad impacts. (1) There would be a pre-phase for deployment where the infrastructure would have to be built into the existing facility, causing delay and in some cases downtime for the existing facility. (2) Lower ROI (Return On Investment) for the customer as one way or another they would end up paying for the change in infrastructure. (3) A much harder buying decision for the customer both because of a longer deployment delay and because of additional cost.

Your robot business will be much simpler if you don’t need changes in infrastructure.

3. But There Is Good News On Infrastructure

As technology advances we humans build new infrastructure for us. Often that same infrastructure turns out to be useful for robots.

When people first had autonomous vehicles driving on freeways (back in the 1980’s!! — see the work of Ernst Dickmanns) there was not much external infrastructure, besides white lines painted on the roads, to help the vehicles. But since then a lot has changed, with new infrastructure that has been installed to help human drivers. We now have GPS, and digital maps, so that our human driven cars know exactly where they are on those maps, and can display it to us, and search the maps for recommended routes. In addition there is congestion information that gets sent to our human driven cars and our phones, which let our route planners suggest faster routes. And the systems in some cars know the speed limits everywhere without having to read the speed limit signs. All these capabilities make it much easier for robot cars, autonomous vehicles, to be able to drive around. That human assist infrastructure has been a real boon towards getting to autonomous vehicles.

In the early days of robots for hospitals and for hospitality (i.e., hotels) and for managed care facilities for the elderly, it was first necessary to install Wi-Fi in the buildings as it was needed for the robots to know what to do next. That was a big barrier to entry for those robots.  Now, however, the humans in those environments expect to have Wi-Fi available everywhere. It started to be deployed in hospitals to support the handheld devices supplied to doctors and nurses so they could record notes and data, but now patients and their visitors expect it too. The same with hotel guests. So requiring Wi-Fi for robots is no longer that much of a barrier to adoption.

In warehouses and factories the hand held bar code scanners have forced the introduction of Wi-Fi there. And in order to make it easy for humans to push around carts, as they do in both those environments, the floors are now generally trash free, and the slopes and lips in the floors are all quite small–infrastructure introduced for the benefit of humans. But this infrastructure, along with plenty of lighting for humans so they can see, make it much easier to introduce robots than it would have been a few decades ago, even if we had had robots back then.

Look for human driven infrastructure changes in whatever domain you want to have your robots work, and see if the robots can make use of that infrastructure. It will likely make them better.

4. deployed Robots Live In A Sea Of Real World Data

The last few years have seen a clear rise in the utility of machine learning. (Just for reference the term “machine learning” first appeared in 1959 in a paper on checkers written by Arthur Samuel; I wrote a really quite bad master’s thesis on machine learning back in 1977.) The web has provided massive amounts of text for machine learning which has given rise to LLMs. It has also supplied images and movies but they, and other images and movies explicitly collected for the purpose, have required human labeling. Recently people have had sensors attached to them and have been asked to do physical tasks to provide data for research projects (they are “research” projects even if done in well funded companies, as this data has not yet led to deployed robots) aiming to use machine learned control systems for robots (it is not clear to me that the type of data currently collected, nor the assumptions about the form of those control systems, will lead to useful results).

If you have deployed robots then data collection with labels may be available to you pretty much for free.

If your robots have cameras on them, then once they are deployed they are able to observe lots of real world things happening around them. If they are doing real work, then those video feeds are synchronized to what is happening in the world. and to external data interactions that the robot has with the work management system installed in the workplace. Thus the visual data is pre-labelled in some way, and can be used to feed learning algorithms with vast quantities of data over time.

Here is an example of some collected data that is used to feed learning algorithms at Robust AI.

Get your robots deployed then have them learn from real world data to improve and extend their performance.

I have recently blogged about my three laws of robotics. Here I talk about my three laws of Artificial Intelligence, about how people perceive AI systems, about how they operate in the world and how difficult it is to make them general purpose in any sense.

1. When an AI system performs a task, human observers immediately estimate its general competence in areas that seem related. Usually that estimate is wildly overinflated.
2. Most successful AI deployments have a human somewhere in the loop (perhaps the person they are helping) and their intelligence smooths the edges.
3. Without carefully boxing in how an AI system is deployed there is always a long tail of special cases that take decades to discover and fix. Paradoxically all those fixes are AI-complete themselves.

I very briefly elaborate on these three laws.

Over Estimating Competence

I talked about  the tendency people have to extrapolate general competence in some area from an observation of a more specific performance in my blog post titled The Seven Deadly Sins of Predicting the Future of AI:

People hear that some robot or some AI system has performed some task. They then take the generalization from that performance to a general competence that a person performing that same task could be expected to have. And they apply that generalization to the robot or AI system.

The competence of AI systems tends to be very narrow and we humans don’t have the right model for estimating that competence. The overhyping of LLMs in the last two years is a case in point.  LLMs cannot reason at all, but otherwise smart people are desperate to claim that they can reason.  No, you are wrong. LLMs do not reason, by any reasonable definition of reason. They are not doing what humans are doing when we say they are reasoning, and applying that word to LLMs saying it is “reasoning but different” simply leads to gross failures in predicting how well the technology will work.

Person In the loop

People get further confused by the fact that there is usually a person in the loop with deployed AI systems. This can happen in two different ways:

1. The person who is using the system bears some of the responsibility for what the system does and consciously, or unconsciously, corrects for it. In the case of search engines, for instance, the AI system offers a number of search results and the person down selects and does the final filtering on the results. The search engine does not have to be as intelligent as if it was going to apply the results of its search directly to the real world. In the case of Roombas there is a handle on the robot vacuum cleaner and the owner steps in and gets the robot out of trouble by picking it up and moving it.
2. The company that is deploying the AI system is operating on a “fake it until you make it” strategy, and there is a person involved somewhere but that fact is deliberately obfuscated.  It turns out that both of the large scale deployments of autonomous vehicles in San Francisco had and have remote people ready to invisibly help the cars get out of trouble, and they are doing so every couple of minutes. A major online seller recently shut down its scanless supermarkets where a customer could walk in, pick up whatever they wanted and walk out and have their credit card charged the correct amount. Every customer was consuming an hour or more of remote humans (in India as it happened) watching and rewatching videos of the customer to determine what it was they had put in their shopping baskets. Likewise for all the campus delivery robot systems–there the companies are selling Universities the appearance of being at the forefront of technology adoption, and not actually providing a robot service at all.

The Long Tail

Claude Shannon first outlined how a computer might be programmed to play chess in an article in Scientific American back in February 1950. He even suggested some mechanisms that might be used to make it learn to play better. In 1959 Arthur Samuel described an implemented program that worked that way and used those learning methods to play the game of Checkers. It was the first time the phrase “machine learning” was used in print. His Checkers program went on to become quite good and to play at an expert human level, even with the tiny computational power of the early 1960s.

But Chess turned out to be much harder. Decades of work went into both improving the learning capabilities of chess and making the look ahead search of possible moves push out further and further. Moore’s Law was the winner and by the 1990s a program had beaten Garry Kasparov, the world champion. There was a lot of controversy over whether the program, Deep Blue from IBM, was a general purpose chess program or a dedicated Kasparov style play beater. By the early 2000s the doubts were gone. Chess programs had gotten good enough to beat any human player.

Chess programs got rid of having to deal individually with special cases through brute force. But Chess is a perfect information game, with no uncertainties and no surprises on the possible pieces that can be on a chess board. It is a closed world.

The places where we really want AI to work well or in more general open worlds.  Our roads where cars drive are an open world. There can be all sorts of things that happen infrequently but need to be handled when they do happen even if the circumstances are different from every thing experienced by AI driving systems before. There are tornados, blizzards, hurricanes, wind borne tumble weeds, and plastic bags, and sheets of styrofoam, and a million other things we could enumerate. Humans use general capabilities, non-driving capabilities, to figure out what to do in unexpected circumstances when driving. Unless an AI system has such general capabilities, every one of the million “long tail events” needs to be trained for.

The things we are asking, or believing the hype about, our AI systems to do have lots and lots of special cases that are not subject to the simple brute force of increasing compute power (despite the hucksters claiming precisely that — just give me $10 billion dollars more of cloud services and our system will learn it all).

No, instead we need to put boxes around our AI systems and products and control where they are applied. The alternative is to have brittle systems that will end up causing economic and perhaps safety upheavals.

Here are some of the things I’ve learned about robotics after working in the field for almost five decades.  In honor of Isaac Asimov and Arthur C. Clarke, my two boyhood go-to science fiction writers, I’m calling them my three laws of robotics.

1. The visual appearance of a robot makes a promise about what it can do and how smart it is. It needs to deliver or slightly over deliver on that promise or it will not be accepted.
2. When robots and people coexist in the same spaces, the robots must not take away from people’s agency, particularly when the robots are failing, as inevitably they will at times.
3. Technologies for robots need 10+ years of steady improvement beyond lab demos of the target tasks to mature to low cost and to have their limitations characterized well enough that they can deliver 99.9% of the time. Every 10 more years gets another 9 in reliability.

Below I explain each of these laws in more detail.  But in a related post here are my three laws of Artificial Intelligence.

Note that these laws are written from the point of view of making robots work in the real world, where people pay for them, and where people want return on their investment. This is very different from demonstrating robots or robot technologies in the laboratory.

In the lab there is phalanx of graduate students eager to demonstrate their latest idea, on which they have worked very hard, to show its plausibility. Their interest is in showing that a technique or technology that they have developed is plausible and promising. They will do everything in their power to nurse the robot through the demonstration to make that point, and they will eagerly explain everything about what they have developed and what could come next.

In the real world there is just the customer, or the employee or relative of the customer. The robot has to work with no external intervention from the people who designed and built it. It needs to be a good experience for the people around it or there will not be more sales to those, and perhaps other, customers.

So these laws are not about what might, or could, be done. They are about real robots deployed in the real world. The laws are not about research demonstrations. They are about robots in everyday life.

The PROMISE GIVEN BY APPEARANCE

My various companies have produced all sorts of robots and sold them at scale. A lot of thought goes into the visual appearance of the robot when it is designed as that tells the buyer or user what to expect from it.

The Roomba, from iRobot, looks like a flat disk. It cleans floors. The disk shape was so that it could turn in place without hitting anything it wasn’t already hitting. The low profile of the disk was so that it could get under the toe kicks in kitchens and clean the floor that is overhung just a little by kitchen cabinets.  It does not look like it can go up and down stairs or even a single step up or step down in a house and it cannot. It has a handle, which make it look like it can be picked up by a person, and it can be. Unlike fictional Rosey the Robot it does not look like it could clean windows, and it cannot. It cleans floors, and that is it.

The Packbot, the remotely operable military robot, also from iRobot, looked very different indeed. It has tracked wheels, like a miniature tank, and that appearance  promises anyone who looks at it that it can go over rough terrain, and is not going to be stopped by steps or rocks or drops in terrain. When the Fukushima disaster happened, in 2011, Packbots were able to operate in the reactor buildings that had been smashed and wrecked by the tsunami, open door handles under remote control, drive up rubble covered staircases and get their cameras pointed at analog pressure and temperature gauges so that workers trying to safely secure the nuclear plant had some data about what was happening in highly radioactive areas of the plant.

The point of this first law of robotics is to warn against making a robot appear more than it actually is.  Perhaps that will get funding for your company leading investors to believe that in time the robot will be able to do all the things its physical appearance suggests it might be able to do. But it is going to disappoint customers when it cannot do the sorts of things that something with that physical appearance looks like it can do.  Glamming up a robot risks overpromising what the robot as a product can actually do.  That risks disappointing customers. And disappointed customers are not going to be an advocate for your product/robot, nor be repeat buyers.

Preserving People’s Agency

The worst thing for its acceptance by people that a robot can do in the workplace is to make their jobs or lives harder, by not letting them do what they need to do.

Robots that work in hospitals taking dirty sheets or dishes from a patient floor to where they are to be cleaned are meant to make the lives of the nurses easier. But often they do exactly the opposite. If the robots are not aware of what is happening and do not get out of the way when there is an emergency they will probably end up blocking some life saving work by the nurses–e.g., pushing a gurney with a critically ill patient on it to where they need to be for immediate treatment. That does not endear such a robot to the hospital staff. It has interfered with their main job function, a function of which the staff is proud, and what motivates them to do such work.

A lesser, but still unacceptable behavior of robots in hospitals, is to have them wait in front of elevator doors, central, and blocking for people. It makes it harder for people to do some things they need to do all the time in that environment–enter and exit elevators.

Those of us who live in San Francisco or Austin, Texas, have had first hand views of robots annoying people daily for the last few years. The robots in question have been autonomous vehicles, driving around the city with no human occupant. I see these robots every single time I leave my house, whether on foot or by car.

Some of the vehicles were notorious for blocking intersections, and there was absolutely nothing that other drivers, pedestrians, or police could do. We just had to wait until some remote operator hidden deep inside the company that deployed them decided to pay attention to the stuck vehicle and get it out of people’s way. Worse, they would wander in to the scene of a fire where there were fire trucks and fire fighters and actual buildings on fire, get confused and just stop, sometime on top of the fire hoses.

There was no way for the fire fighters to move the vehicles, nor communicate with them. This is in contrast to an automobile driven by a human driver. Fire fighters can use their normal social interactions to communicate with a  driver, and use their privileged position in society as front line responders to apply social pressure on a human driver to cooperate with them. Not so with the autonomous vehicles.

The autonomous vehicles took agency from people going about their regular business on the streets, but worse took away agency from firefighters whose role is to protect other  humans. Deployed robots that do not respect people and what they need to do will not get respect from people and the robots will end up undeployed.

Robust ROBOTS THAT WORK EVERYTIME

Making robots that work reliably in the real world is hard. In fact, making anything that works physically in the real world, and is reliable, is very hard.

For a customer to be happy with a robot it must appear to work every time it tries a task, otherwise it will frustrate the user to the point that they will question whether it make their life better or now.

But what does appear mean here? It means that the user can have the assumption that it going to work, as their default understanding of what will happen the world.

The tricky part is that robots interact with the real physical world.

Software programs interact with a well understood abstracted machine, so they tend not fail in a manner where the instructions in them do not get executed in a consistent way by the hardware on which they are running.  Those same programs may also interact with the physical world, be it a human being, a network connection, or an input device like a mouse. It is then that the programs might fail as the instructions in them are based on assumptions in the real world that are not met.

Robots are subject to forces in the real world, subject to the exact position of objects relative to them, and subject to interacting with humans who are very variable in their behavior. There are no teams of graduate students or junior engineers eager to make the robot succeed on the 8,354^th attempt to do the same thing that has worked so many times before.  Getting software that adequately adapts to the uncertain changes in the world in that particular instance and that particular instant of time is where the real challenge arises in robotics.

Great looking videos are just not the same things as working for a customer every time.  Most of what we see in the news about robots is lab demonstrations. There is no data on how general the solution is, nor how many takes it took to get the video that is shown. Even worse sometimes the videos are tele-operated or sped up many times over.

I have rarely seen a new technology that is less than ten years out from a lab demo make it in to a deployed robot. It takes time to see how well the method works, and to characterize it well enough that it is unlikely to fail in a deployed robot that is working by itself in the real world. Even then there will be failures, and it takes many more years of shaking out the problem areas and building it into the robot product in a defensive way so that the failure does not happen again.

Most robot require kill buttons or estops on them so that a human can shut them down. If a customer ever feels the need to hit that button then the people who have built and sold the robot have failed. They have not made it operate well enough that the robot never gets into a state where things are going that wrong.

There are three things that the existence of LLMs, such as ChatGPT-3.5 and ChatGPT-4 make us have to rethink. At different times and amongst different communities they have all had lots of AI researchers talking about them, often with much passion.

Here are three things to note:

1. The Turing Test has evaporated.
2. Searle’s Chinese Room showed up, uninvited.
3. Chomsky’s Universal Grammar needs some bolstering if it is to survive.

We’ll talk about each in turn.

The Turing Test

In a 1950 paper titled Computing Machinery and Intelligence, Alan Turing used a test which involved a human deciding whether an entity that the person was texting with was a human or a computer. Of course, he did not use the term “texting” as that had not yet been invented, rather he suggested that the communication was via a “teleprinter”, which did exist at the time, where the words typed in one location appeared on paper in a remote location. “Texting” is the modern equivalent.

Turing used this setup as rhetorical device to argue that if you could not accurately and reliably decide whether it was a person or a computer at the other end of the line then you had to grant that a machine could be intelligent.  His point was that it was not just simulating intelligence but that it would actually be intelligent if people could not tell the difference.

Turing said;

I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 10⁹, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent, chance of making the right identification after five minutes of questioning.

His number 10⁹ referred to how many bits of program would be needed to achieve this result, which is 125MB, i.e., 125 Mega Bytes. Compare this with ChatGPT-3.5 which has 700GB, or 700 Giga Bytes, of weights (175 billion 32 bit weights) that it has learned, which is almost 6,000 times as much.

His paragraph above continues:

The original question, ‘Can machines think!’ I believe to be too meaningless to deserve discussion. Nevertheless I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. 

Despite his goal to be purely a rhetorical device to make the question, ‘Can machines think!’ (I assume the punctuation was a typo and was intended to be ‘?’) meaningless, this led to people calling the machine/person discernment test the Turing Test, and it became the default way of thinking about how to determine when general Artificial Intelligence had been achieved.  But, of course, it is not that simple. That didn’t stop annual Turing Tests being set up, with entrants from mostly amateur researchers, who had built chat bots designed not to do any useful work in the world, but designed and built simply to try to pass the Turing Test. It was a bit of a circus and mostly not very useful.

Earlier this year I felt like I was not hearing about the Turing Test with regards to all the ChatGPTs, and in fact the scientific press had noticed this too, with this story in Nature in July of this year:

Don’t worry, there are still papers being written on the Turing Test and ChatGPT, for instance this one from October 2023, but the fervor of declaring that it is important has decreased.

We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4 prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and GPT-3.5 (14%), but falling short of chance and the baseline set by human participants (63%).

In general the press has moved away from the Turing Test. ChatGPT seems to have the sort of language expertise that people imagined some system as intelligent as a person would have, but it has become clear that it is not the crystalline indication of intelligence that Turing was trying to elucidate.

SEARLE’S CHINESE ROOM

In 1980, John Searle, a UC Berkeley philosopher, introduced the idea of a “Chinese Room”, as a way to argue that computers could not be truly intelligent in the way that people are, not truly engaged with the world in the way people are, and not truly sentient in the way people are.

He chose “Chinese” as the language for the room as it was something totally foreign to most people working in Artificial Intelligence in the US at the time. Furthermore its written form was in atomic symbols.

Here is what ChatGPT-3.5 said when I asked it to describe Searle’s Chinese Room. I have highlighted the last clause in blue.

Imagine a person who doesn’t understand the Chinese language locked inside a room. This person has a set of instructions (a program) written in English that tells them how to manipulate Chinese symbols based on the symbols that are input to the room through a slot. They have no understanding of Chinese and don’t know the meanings of the symbols.

From the outside, someone passes messages written in Chinese through the slot into the room. The person inside the room follows the instructions in the program and produces responses in Chinese based solely on symbol manipulation, without understanding the content or meaning of the messages.

To an observer outside the room, it may appear as though the person inside understands Chinese, as they can generate coherent responses in Chinese. However, the person inside the room doesn’t actually understand the language. They are merely following a set of syntactic rules or algorithms for manipulating symbols, without grasping the semantics or meanings of those symbols.

To prove that ChatGPT really is the Chinese Room, I typed in some Chinese text to version 3.5 and it responded with Chinese text:

艾未未是谁？

艾未未是一位知名的中国艺术家、社会活动家和评论家。他以其大胆的艺术创作和公开对政府审查和人权侵犯的批评而闻名。艾未未的作品涉及雕塑、摄影、行为艺术、装置艺术和社交媒体等多种媒介，展现了对政治、社会和人权议题的关注。他因批评中国政府而多次引起争议，这导致了他与政府当局的冲突，并且在过去曾经被拘留和监禁。他因其独特的艺术作品和公开表达的观点而在国际艺术界和社会活动家圈中备受瞩目。

My question was “Who is Ai Wei Wei?” and it responded in what people tell me is perfectly good Chinese, describing the Chinese artist Ai Wei Wei, his art, and his run ins with the government.

So here we have Searle’s Chinese Room.  It has been trained on billions of words from the web but has no experience of the world, has never heard Chinese spoken, and has never even read Chinese characters as characters but rather has seen them only as integers along with (I assume) a pre-processing step to map whichever of the five (Unicode, GB, GBK, Big 5, and CNS) common digital Chinese character code standards each document uses to that common set of integers. (The fact that ChatGPT `knows’, in its description of the room, above, that Chinese is written in symbols is not because it has ever seen them, but because it has “read” that Chinese uses symbols.)

The fact that GPT is the Chinese Room, the fact that one now exists, means that many of the old arguments for and against Searle’s position that he was staking out with his rhetorical version of the room must now be faced squarely and perhaps re-evaluated.  Searle’s Chinese Room was a topic of discussion in AI for well over 25 years. Everyone had to have an opinion or argument.

In my book Flesh and Machines: How Robots Will Change Us (Pantheon, New York, 2002), I made two arguments that were in opposition to Searle’s description of what his room tells us.

Firstly, I argued (as did many, many others) that indeed Searle was right that the person in the room could not be said to understand Chinese. Instead we argued that it was the whole system, the person, the rule books, and the state maintained in following the rules that was what understood Chinese. Searle was using the person as a stand in for a computer fresh off the production line, and ignoring the impact of loading the right program and data on to it. In the ChatGPT case it is the computer, plus the algorithms for evaluating linear neuron models plus the 175 billion weights that are together what make ChatGPT-3.5 understand Chinese, if one accepts that it does. In my book I said that no individual neuron in a human brain can be said to understand Chinese, it has to be the total system’s understanding that we talk about. ChatGPT-3.5 is an example of a computer doing the sort of thing that Searle was arguing was not possible, or at least should not be spoken about in the same way that we might speak about a person understanding Chinese.

Secondly, I argued (using Searle as the person in the room as he sometimes did):

Of course, as with many thought experiments, the Chinese room is ludicrous in practice. There would be such a large set of rules, and so many of them would need to be followed in detailed order that Searle would need to spend many tens of years slavishly following the rules, and jotting down notes on an enormous supply of paper. The system, Searle and the rules, would run as a program so slowly that it, the system, could not be engaged in any normal sorts of perceptual activity. At that point it does get hard to effectively believe that the system understands Chinese for any usual understanding of `understand’. But precisely because it is such a ludicrous example, slowed down by factors of billions, any conclusions from that inadequacy can not be carried over to making conclusions about whether a computer program running the same program `understands’ Chinese.

Hmm, well my bluff has been call by the existence of ChatGPT.  First, note that I was right about the size of the rule set, 175 billion neural weights, that it would take a person effectively forever to follow them. But every modern laptop can hold all those rules in the file system (it is less than a terabyte of memory), and the algorithm is parallel enough that a chunk of processing in the cloud can make ChatGPT run at human language speeds.

If I maintain my above argument from 2002, I would have to say that ChatGPT does `understand’ Chinese. But those who have read my writings over the years would guess, rightly, that I don’t think it does. Without grounding in physical reality I don’t think a machine can understand in the same way we do.  ChatGPT is just like someone following rules with no semantic understanding of the symbols, but it does it at the speed my argument above said was necessary for it to really be understanding.  But now I’m going to say it still doesn’t understand. My old self and my today self are not being intellectually coherent, so I am going to have to think about this some more over the next few years and refine, perhaps rethink, but certainly change in some way what it is I conclude from both Searle and ChatGPT existing.

Other people over the last forty years have argued, and I have agreed, that language in humans is strongly grounded in non-language. So, we have argued that a computer program, like ChatGPT-3.5 could not have a consistent performance level that would seem like human language.  ChatGPT-3.5 certainly seems to have such consistent performance, as long as you don’t poke it too deep–it certainly has a level that would work for most of your daily interactions with strangers.  Our arguments are therefore challenged or broken. I don’t yet know how to fix them.

CHOMSKY’S UNIVERSAL GRAMMAR

Nom Chomsky is the most influential linguist ever and he spent most of his career in MIT’s linguistics department. His advocacy of the existence of a universal grammar system in humans, and humans only, has shaped much cognitive science and theories about cognition in both humans and non-humans.

I asked ChatGPT to “explain Chomsky’s universal grammar”, and it replied:

Generative Pre-trained Transformer models (GPTs) are now all the rage and have inspired op-eds being written by everyone from Henry Kissinger (WSJ) to Noam Chomsky (NYTimes) in just the last month. That sure is some hype level.

Way back in the early history of GPTs, January 1^st this year, I wrote briefly about them and said:

Calm down people. We neither have super powerful AI around the corner, nor the end of the world caused by AI about to come down upon us.

I stick with that advice, but in this post I want to say why, and talk about where these systems will have impact. In short, there will be valuable tools produced, and at the same time lots of damaging misuse.

What triggers me to write here in more detail is both the continued hype, and the release of GPT-4 during the week of March 13th, and the posting of the “GPT-4 Technical Report” by many hundreds of authors at OpenAI.  [[The linked PDF is 98 pages long and contains two papers, one titled “GPT-4 Technical Report” that fills the first 38 pages of the PDF, and one titled “GPT-4 System Card” which is 60 pages long with its pages number 1 to 60, but mapped to PDF pages 39 to 98.]]

In mid-February of this year Stephen Wolfram wrote a very clear (it is long, this is a hard and big topic) post about how and why ChatGPT works. As he says, it is “Just Adding One Word at a Time”. [[Actually, in the couple of days I have been writing my post here, Wolfram’s post has also come out as a printed book…]]

Together, the OpenAI and Wolfram reports give a very good technical understanding of most things GPT.

state of the art GPTs from Open AI

For the last few months there has been lots of excitement about the 175 billion parameter GPT-3 from the company Open AI. It was set up, under the name ChatGPT, so that people could query it, type in a few words and have it “answer” the question. The words set the context and then one word at a time it pops out the word to follow what the context, now including what it had already said, that its learned model judged to be a good follow on word. There is some randomness in choosing among competing very good words, so it answers questions differently at different times. Microsoft attached GPT to its search engine Bing at around the same time.

Sometimes the results seem stunningly good, and people of all stripes have jumped to the conclusion that GPT-3 was heralded the coming of “Artificial General Intelligence”. [[By the way, even since the earliest days of AI, the 1955 proposal for the 1956 workshop on AI, the document in which the term AI first appears anywhere, the goal of the researchers was to produce general intelligence. That AGI is a different term than AI now is due to a bunch of researchers a dozen or so years ago deciding to launch a marketing campaign for themselves by using a new buzz acronym. “AGI” is just “AI” as it was known for the first 50+ years of its existence. Hype produced the term “AGI” with which we are now saddled.]]

This inference of AI arriving momentarily is a clear example of how people mistake performance for competence.  I talked about it back in 2017 as one of the seven deadly sins of predicting the future of AI. I said then that:

We [humans] are able to generalize from observing performance at one task to a guess at competence over a much bigger set of tasks. We understand intuitively how to generalize from the performance level of the person to their competence in related areas.

But the skills we have for doing that for a person break down completely when we see a strong performance from an AI program. The extent of the program’s competence may be extraordinarily narrow, in a way that would never happen with a person.  I think that a lot of people, early on certainly, made this mistake with GPT-3. I’ll show some examples of how GPT-3 and GPT-4 fail in unintuitive ways below.

Meanwhile, in the week of March 13^th, 2023, GPT-4, with a reported 100 trillion learned parameters (i.e., about 571 times as many as for GPT-3), was introduced. And it turned out, according to Peter Lee at Microsoft, that the GPT version that had been part of Bing for the previous weeks and months was GPT-4 all along.

According to reports GPT-4 felt qualitatively different to many users. It felt “more”, and “better”. This sort of feeling something has changed has been reported in the 1990’s when people played a chess program with a previously unachievable depth limit, or when working with a theorem proving program when it can search further than before. (Refs, which will take me longer to track down are to Garry Kasparov and Bob Constable (of Cornell).) I think the “this is better” feeling is natural here, but the presence of that feeling is not in itself a particularly useful indicator.

SOme Dumb things from CHATGPT

GPT-n cannot reason, and it has no model of the world. It just looks at correlations between how words appear in vast quantities of text from the web, without know how they connect to the world. It doesn’t even know there is a world.

I think it is a surprise to many that it seems as smart as it does given that all it has is these correlations. But perhaps that says something about how we humans relate to language and the world, and that language really only, perhaps, has a tenuous connection to the world in which we live; a scary thought.

There is a veritable cottage industry of individuals showing how LLMs can easily be provoked into showing that they have no capability in spatial reasoning, ordinal numbers, or even small quantities below 10.

Here are examples of troubles with numbers, and other interesting failures, with ChatGPT in the Communications of the ACM from earlier this year. The authors, Gary Marcus and Ernie Davis, are both faculty at NYU. Here is just one of their examples:

Here is an example of complete lack of spatial reasoning from an article by Fletcher Jones Professor of Economics Gary N. Smith of Pomona College:

Human: The table didn’t fit in the car because it was too small. What was too small? 

ChatGPT:  The table was too small to fit in the car.

Human: Would it help to get a smaller car? 

ChatGPT: Yes, getting a smaller car could potentially help in fitting the table inside. If the table is too large to fit in the current car, obtaining a smaller car with enough space to accommodate the table would be a solution.

Here is another example which Marcus and Davis write about, where someone asked ChatGPT to write a program.

It writes a Python program without getting the indentation right, and without actually addressing the JSON description part of the specification, but it sure manages to reproduce the sexism and racism contained in its training set. See the guardrails in GPT-4 below.

And here are three examples from the Marcus and Davis paper above (all due to other authors who they cite), where ChatGPT is happy to make stuff up, because it really doesn’t understand how important many words really are:

Two Simple But Amazing examples

Here are two tweets that random people sent out where ChatGPT seems to be funny and smart, and where it doesn’t have to reason to get there, rather it has to generate plausible text. And these two simple examples show how plausible it can be.

And this one runs up against some guard rails that have been put into the system manually, but which are bust through on the second request.

I think it is easy to see from these two examples that performance/competence confusion is very likely.  It makes no sense that a person who could respond in these ways would be as ritualistically as dumb about numbers and spatial relations as the previous section reveals for ChatGPT.

What does OPEN AI Say about GPT-4?

The opening section of the GPT-4 Tech Report from Open AI is instructive, as it includes this paragraph (my emphasis):

Despite its capabilities, GPT-4 has similar limitations to earlier GPT models [1, 31, 32]: it is not fully reliable (e.g. can suffer from “hallucinations”), has a limited context window, and does not learn from experience. Care should be taken when using the outputs of GPT-4, particularly in contexts where reliability is important.

Open AI is being quite clear here that GPT-4 has limitations.  However they appear to be agnostic on who should be taking care.  Is it the responsibility of people who use the GPT-4 parameter set in some product, or is it the end user who is exposed to outputs from that product? Open AI does not express an opinion on this matter.

In a second paper in the .pdf, i.e., the “GPT-4 System Card” they go through the mitigation against dangerous or wrong outputs that they have worked on for the last six months, and give comparisons between what was produced early on at “GPT-4 (early)” and what is produced now at “GPT-4 (launch)”. They have put in a number of guard rails that clearly reduce the amount of both objectionable and dangerous output that can be produced. Nevertheless on page 19 of the System Card (page 57 of the .pdf) they say:

As noted above in 2.2, despite GPT-4’s capabilities, it maintains a tendency to make up facts, to double-down on incorrect information, and to perform tasks incorrectly. Further, it often exhibits these tendencies in ways that are more convincing and believable than earlier GPT models (e.g., due to authoritative tone or to being presented in the context of highly detailed information that is accurate), increasing the risk of overreliance.

This is pretty damning. Don’t rely on outputs from GPT-4.

Earlier in the System Card report (page 7/45):

In particular, our usage policies prohibit the use of our models and products in the contexts of high risk government decision making (e.g, law enforcement, criminal justice, migration and asylum), or for offering legal or health advice.

Here they are protecting themselves by outlawing certain sorts of usage in their license.

This is in the context of their human red team having probed GPT-4 and introduced new training so that often it will refuse to produce harmful text when it matches a class of prompts against which it has been trained.

But their warnings reproduced above  say that they are not at all confident that we will not see real problems with some of the things produced by GPT-4. They have not been able to bullet-proof it with six months of work by a large team.  This is no surprise. There are many many long tail cases to consider and patch up. The same was true for autonomous driving and the result is that we are three to five years on from where executives at major automobile companies were predicting we would have level 4 driving in consumer cars. That experience should be a cautionary tale for GPT-4 and brethren, saying that reliance on them will be fraught for many years to come, unless they are very much boxed in to how they can be used.

On March 21^st, 2023, Sundar Pichai, CEO of Google, on the introduction of Bard A.I., Google’s answer to GPT-4, warned his employees, that “things will go wrong”.

Always a person in the loop in successful AI Systems

Many successful applications of AI have a person somewhere in the loop.  Sometimes it is a person behind the scenes that the people using the system do not see, but often it is the user of the system, who provides the glue between the AI system and the real world.

This is true of language translation systems where a person is reading the output and, just as they do with children, the elderly, and foreigners, adapts quickly to the mistakes the person or system makes, and fill in around the edges to get the meaning, not the literal interpretation.

This is true of speech understanding systems where we talk to Alexa or Google Home, or our TV remote, or our car. We talk to each of them slightly differently, as we humans quickly learn how to adapt to their idiosyncracies and the forms they can understand and not understand.

This is true of our search engines, where we have learned how to form good queries that will get us the information we actually want, the quickest.

This is true our smart cameras where we have learned how to take photographs with them rather than with a film camera (though in this case they are often superhuman in their capabilities).

This is true where we are talking to a virtual agent on a web site where we will either be left to a frustrating experience or the website has back up humans who connect in to help with tricky situations.

This is true of driver assist/self driving modes in cars where the human driver must be prepared to take over in an instant in high stress situations.

This is true of mobile robots in hospitals, taking the dirty sheets and dishes to be cleaned, or bringing up prescriptions from the hospital pharmacy,  where there is a remote network operations center that some unseen user is waiting to take over control when the robot gets confused.

This is true of chess where the best players are human chess experts working with a chess engine, and together they play better than any chess engine by itself.

And this is true of art work, produced by stable diffusion models, where the eye of the beholder always belongs to a human.

Below I predict the future for the next few years with GPTs and point out that their successful deployment will always have a person in the loop in some sense.

Predicting the future is hard

Roy Amara, who died on the last day of 2007, was the president of a Palo Alto based think tank, the Institute for the future, and is credited with saying what is now known as Amara’s Law:

We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.

This has been a common problem with Artificial Intelligence, and indeed of all of computing. In particular, since I first became conscious of the possibility of Artificial Intelligence around 1963 (and as an eight year old proceeded to try to build my own physical and intelligent computers, and have been at it ever since), I have seen these overestimates many many times.

A few such instances of AI technologies that have induced gross overestimates of how soon we would get to AGI, in roughly chronological order, that I personally remember include:

John McCarthy’s estimate that the computers of the 1960’s were powerful enough to support AGI, Minsky and Michie and Nilsson each believing that search algorithms were the key to intelligence, neural networks (volume 3, perceptrons) [[I wasn’t around for the first two volumes; McCulloch and Pitts in 1943, Minsky in 1953]], first order logic, resolution theorem proving, MacHack (chess 1), fuzzy logic, STRIPS, knowledge-based systems (and revolutionizing medicine), neural networks (volume 4, back propagation), the primal sketch, self driving cars (Dickmanns, 1987), reinforcement learning (rounds 2 and 3), SOAR, qualitative reasoning, support vector machines, self driving cars (Kanade et al, 1997), Deep Blue (chess 2), self driving cars (Thrun, 2007), Bayesian inference, Watson (Jeopardy, and revolutionizing medicine), neural networks (volume 5, deep learning), Alpha GO, reinforcement learning (round 4), generative images, and now large language models. All have heralded the imminence of human level intelligence in machines. All were hyped up to the limit, but mostly in the days when very few people were even aware of AI, so very few people remember the levels of hype. I’m old. I do remember all these, but have probably forgotten quite a few…

None of these things have lived up to that early hype. As Amara predicted at first they were overrated. But at the same time, almost every one of these things have had long lasting impact on our world, just not in the particular form that people first imagined. As we twirled them around and prodded them, and experimented with them, and failed, and retried, we remade them in ways different from how they were first imagined, and they ended up having bigger longer term impacts, but in ways not first considered.

How does this apply to GPT world?  As always, the hype is overestimating the utility and the threats. However much will come from GPT-like systems.

Do I have it wrong?

Ada Lovelace said something similar to Amara’s Law back in 1843. This is from her first paragraph of “Note G”, in her notes she wrote to accompany a translation she made of someone else’s notes on the Analytical Engine in 1843. With her emphasis:

In considering any new subject, there is frequently a tendency, first, to overrate what we find to be already interesting or remarkable; and, secondly, by a sort of natural reaction, to undervalue the true state of the case, when we do discover that our notions have surpassed those that were really tenable.

Here the first half matches the first half of Amara’s Law. Her second half touches on something different than Amara’s second half. She says that when we get chastened by discovering we were overly optimistic out of the gate we pull back too far on our expectations.

Having seen the hype cycle so often and seen it go a particular way so often, am I now undervaluing the subject of a new hype cycle? If this is hype cycle n, I would have been right to undervalue the hype for the previous n-1 times. Am I just pattern matching and thinking it would be right to undervalue for time n? Am I suffering from cynicism?  And I just a grumpy old guy who thinks he’s seen it all? Perhaps. We’ll have to see with time.

In General, what will Happen?

Back in 2010 Tim O’Reilly tweeted out “If you’re not paying for the product then you’re the product being sold.”, in reference to things like search engines and apps on telephones.

I think that GPTs will give rise to a new aphorism (where the last word might vary over an array of synonymous variations):

If you are interacting with the output of a GPT system and didn’t explicitly decide to use a GPT then you’re the product being hoodwinked.

I am not saying everything about GPTs is bad. I am saying that, especially given the explicit warnings from Open AI, that you need to be aware that you are using an unreliable system.

Using an unreliable system sounds awfully unreliable, but in August 2021 I had a revelation at TED in Monterey, California, when Chris Anderson (the TED Chris), was interviewing Greg Brockman, the Chairman of Open AI about an early version of GPT. He said that he regularly asked it questions about code he wanted to write and it very quickly gave him ideas for libraries to use, and that was enough to get him started on his project. GPT did not need to be fully accurate, just to get him into the right ballpark, much faster than without its help, and then he could take it from there.

Chris Anderson (the 3D robotics one, not the TED one) has likewise opined (as have responders to some of my tweets about GPT) that using ChatGPT will get him the basic outline of a software stack, in a well tread area of capabilities, and he is many many times more productive than with out it.

So there, where a smart person is in the loop, unreliable advice is better than no advice, and the advice comes much more explicitly than from carrying out a conventional search with a search engine.

[[Earlier this year I posted to my facebook friends that I was having trouble converting over a software system that I have been working on for 30+ years from running natively on an x86 Mac to running natively on an M1 ARM Mac. The issue was that my old technique for changing memory that my compiler had just written instructions into as data to then allow it to be executed as instructions was not working.  John Markoff suggested that I ask ChatGPT, which I then did. It gave me a perfect multi-paragraph explanation of how to do it, starting off with “…on an M1 Macintosh…”. The problem was the explanation was completely accurate for an x86 Macintosh, and was exactly what I had been doing for the last 10+ years, but completely wrong for an M1 Macintosh.]]

The opposite of useful can also occur, but again it pays to have a smart human in the loop.  Here is a report from the editor of a science fiction magazine which pays contributors. He says that from late 2022 through February of 2023 the number of submissions to the magazine increased by almost two orders of magnitude, and he was able to determine that the vast majority of them were generated by chatbots. He was the person in the loop filtering out the signal he wanted, human written science fiction, from vast volumes of noise of GPT written science fiction.

Why should he care?  Because GPT is an auto-completer and so it is generating variations on well worked themes.  But, but, but, I hear people screaming at me.  With more work GPTs will be able to generate original stuff.  Yes, but it will be some other sort of engine attached to them which produces that originality.  No matter how big, and how many parameters, GPTs are not going to to do that themselves.

When no person is in the loop to filter, tweak, or manage the flow of information GPTs will be completely bad. That will be good for people who want to manipulate others without having revealed that the vast amount of persuasive evidence they are seeing has all been made up by a GPT.  It will be bad for the people being manipulated.

And it will be bad if you try to connect a robot to GPT. GPTs have no understanding of the words they use, no way to connect those words, those symbols, to the real world. A robot needs to be connected to the real world and its commands need to be coherent with the real world. Classically it is known as the “symbol grounding problem”. GPT+robot is only ungrounded symbols. It would be like you hearing Klingon spoken, without any knowledge other than the Klingon sound stream (even in Star Trek you knew they had human form and it was easy to ground aspects of their world). A GPT telling a robot stuff will be just like the robot hearing Klingonese.

[[And, of course, for those who have read my more obscure writing for the last 30+  years (see Nature (2001), vol 409, page 409), I do have issues with whether the symbol grounding problem is the right way of thinking about things, but for this argument it is good enough.]]

My argument here is that GPTs might be useful, and well enough boxed, when there is an active person in the loop,  but dangerous when the person in the loop doesn’t know they are supposed to be in the loop. [This will be the case for all young children.]  Their intelligence, applied with strong intellect, is a key component of making any GPT be successful.

Specific Predictions

Here I make some predictions for things that will happen with GPT types of systems, and sometimes coupled with stable diffusion image generation. These predictions cover the time between now and 2030. Some of them are about direct uses of GPTs and some are about the second and third order effects they will drive.

1. After years of Wikipedia being derided as not a referable authority, and not being allowed to be used as a source in serious work, it will become the standard rock solid authority on just about everything. This is because it has built a human powered approach to verifying factual knowledge in a world of high frequency human generated noise.
2. Any GPT-based application that can be relied upon will have to be super-boxed in, and so the power of its “creativity” will be severely limited.
3. GPT-based applications that are used for creativity will continue to have horrible edge cases that sometimes rear their ugly heads when least expected, and furthermore, the things that they create will often arguably be stealing the artistic output of unacknowledged humans.
4. There will be no viable robotics applications that harness the serious power of GPTs in any meaningful way.
5. It is going to be easier to build from scratch software stacks that look a lot like existing software stacks.
6. There will be much confusion about whether code infringes on copyright, and so there will be a growth in companies that are used to certify that no unlicensed code appears in software builds.
7. There will be surprising things built with GPTs, both good and bad, that no-one has yet talked about, or even conceived.
8. There will be incredible amounts of misinformation deliberately created in campaigns for all sorts of arenas from political to criminal, and reliance on expertise will become more discredited, since the noise will drown out any signal at all.
9. There will be new categories of pornography.