Many new technologies have unexpected impacts on the physical or social world in which we live.

When the first IMPs for the fledgling ARPANET were being built starting in 1969 at BBN in Cambridge, MA, I think it safe to say that no one foresaw the devastating impact that the networking technology being developed would have on journalism thirty to fifty years later. Craigslist replaced classified ads in newspapers and took a huge amount of their revenue away, and then Google provided a new service of search for things that one might buy and at the same time delivered ads for those things, taking away much of the rest of advertising revenue from print, radio, and TV, the homes of most traditional journalism. Besides loss of advertising cutting the income stream for journalism, and thus cutting the number of employed journalists, the new avenues for versions of journalism are making it more difficult for traditional print journalists to compete, as John Markoff recently talked about in announcing his retirement from the New York Times.

A way of sharing main frame computer power between research universities ended up completely disrupting how we get our news, and perhaps even who we elected as President.

Where might new unexpected upendings of our lives be coming from?

Perhaps the new technology with the biggest buzz right now is self driving cars.

In this post I will explore two possible consequences of having self driving cars, two consequences that I have not seen being discussed, while various car companies, non-traditional players, and startups debate what level of autonomy we might expect in our cars and when. These potential consequences  are self-driving cars as social outcasts and anti-social behavior of owners.  Both may have tremendous and unexpected influence on the uptake of self-driving cars.  Both are more about the social realm than the technical realm, which is perhaps why technologists have not addressed them. And then I’ll finish, however, by dissing a non-technical aspect of self driving cars that has been overdone by technologists and other amateur philosophers with an all out flame. And yes, I am at best an amateur philosopher too. That’s why it is a flame.

But first…

Levels of Autonomy

There is general agreement on defining different levels of autonomy for cars, numbered 0 through 5, although different sources have slightly different specifications for them.  Here are the levels from the autonomous car entry in Wikipedia which attributes this particular set to the SAE (Society of Automotive Engineers):

• Level 0: Automated system has no vehicle control, but may issue warnings.
• Level 1: Driver must be ready to take control at any time. Automated system may include features such as Adaptive Cruise Control (ACC), Parking Assistance with automated steering, and Lane Keeping Assistance (LKA) Type II in any combination.
• Level 2: The driver is obliged to detect objects and events and respond if the automated system fails to respond properly. The automated system executes accelerating, braking, and steering. The automated system can deactivate immediately upon takeover by the driver.
• Level 3: Within known, limited environments (such as freeways), the driver can safely turn their attention away from driving tasks, but must still be prepared to take control when needed.
• Level 4: The automated system can control the vehicle in all but a few environments such as severe weather. The driver must enable the automated system only when it is safe to do so. When enabled, driver attention is not required.
• Level 5: Other than setting the destination and starting the system, no human intervention is required. The automatic system can drive to any location where it is legal to drive and make its own decision.

Some versions of level 4 specify the that there may be geographical restrictions, perhaps to places that have additional external infrastructure installed.

Today almost all new cars have level 1 autonomy features, and level 2 autonomy is becoming more common in production products. Some manufacturers are releasing software for level 4 though the legality and prudence of doing so right now is open to question.

There is much debate on how to have safe versions of level 2 and level 3 autonomy as both require a human to jump into the control loop when their attention has been wandering.  The time available for the person to reorient their concentration in order to respond to events in the world is often much shorter than what people really need.   I think most people agree that there might be a natural progression from level 4 to level 5, but there are different opinions on whether going from level 2 to level 3, or, more vociferously, from level 3 to level 4 are natural progressions.  As a result there are advocates for going straight to level 4, and there are many start up companies, and non-traditional players (e.g., Google) trying to go directly to level 4 or level 5 autonomy.

The rest of this post is about level 4 and level 5 autonomy.  What are the unexpected social consequences of having cars driving around without a human driver in command or at the ready to be in command?

 

1. Social Outcasts

Suppose you are walking along a twisting narrow country ride at night, with no shoulder and thick vegetation right at the edge of the pavement, and with no moon out, and you hear a car approaching.  What do you do?  I know what I would do!  I’d get off the road, climbing into the bushes if necessary, until the car had passed.  Why would I do that?  Because I would have no indication of whether the driver of the car had seen me and was going to avoid hitting me.

We all realize that on dark narrow country roads anonymous cars are first class citizens and we pedestrians are second class.  We willingly give cars the right of way.

But what about in the daytime (or even night time) in an urban area where you live?  There, pedestrians and cars interact all the time.  And much of that interaction is social interaction between the person on the street and the person behind the wheel.  Sometimes it is one way interaction, but often it is two way interaction.  Two questions arise.  If self driving cars can not participate in these interactions how will people feel about these new aliens sharing their space with them?  And in the interests of safety for pedestrians, how much will the performance of self driving cars need to be detuned relative to human driven cars, and how will that impact the utility of those cars, and degrade the driving experience of people that have traditional level 1 cars?

Within a few blocks of where I live  in Cambridge, MA, are two different microcosms of how people and cars interact.  Other neighborhoods will have other ways of interacting but the important point is how common interaction is.

The streets for a few blocks around where I live are all residential with triple decker apartment buildings or single or duplex houses on small lots.  The streets are narrow, and many of them are one-way.  There are very few marked pedestrian crossings.  People expect to be able to cross a street at any point, but they know there is give and take between drivers and pedestrians and there are many interactions between cars and people walking.  They do not think that cars are first class citizens and that they are second class.  Cars and people are viewed as equals, unlike on a narrow country road at night.

Cars and people interact in three ways that I have noticed in this area.  First, on the longer through roads the cars travel without stop signs, but there are stop signs on the entering or cross side streets.  People expect the right of way on the longer streets too, expecting that cars that have stopped on a side street will let them walk in front if they are about to step off the curb.  But people look to the driver for acknowledgement that they have been seen before they step in front of the car.  Second, when people want to cross a street between intersections or on one of the through streets without stop signs they wait for a bit of a gap between cars, step out cautiously if one is coming and confirm that the car is slowing down before committing to be in the middle of the road.   But often they will step off the curb and partially into the road not expecting the very next car to let them go, but the one that is second to reach where they are–they do expect that second car to let them cross.  And third, the sidewalks are narrow, and especially when there is snow can be hard to walk on (residents are responsible for the sidewalk in front of their properties, and can take a while to clear them) so in winter people often walk along the roads, trying to give room for the cars to go by, but nevertheless expecting the cars to be respectful of them and give them room to walk along the road.

A few blocks further away from where I live is a somewhat different environment, a commercial shopping, bar, and restaurant area (with the upper floors occupied by M.I.T. spin-off startups), known as Central Square. There are marked pedestrian crossings there, and mostly people stick to crossing the roads at those designated places.  Things are a little less civil here, perhaps because more people driving through are not local residents from right around the neighborhood.

People step out tentatively into the marked cross walks and visually check whether on-coming drivers are slowing down, or indicate in some way that they have seen the pedestrian.  During the day it easy to see into the cars and get an idea of what the driver is paying attention to, and the same is actually true at night as there is enough ambient light around to see into most cars.  Pedestrians and drivers mostly engage in a little social interaction, and any lack of interaction is usually an indicator to the pedestrian that the driver has not seen them.  And when such a driver barrels through the crossing the pedestrians get angry and yell at the car, or even lean their hands out in front of the car to show the driver how angry they are.

Interestingly, many pedestrians reward good behavior by drivers.  Getting on the main street or off of the main street from or onto a small side street can often be tricky for a driver.  There are often so many people on the sidewalks that there is a constant flow of foot traffic crossing the exits or entrances of the side streets.   Drivers have to be patient and ready for a long wait to find a break.  Often pedestrians who have seen how patient a driver is being will voluntarily not step into the cross walk, and either with a head or hand signal indicate to a driver that they should head through the crossing.  And if the driver doesn’t respond they make the signal again–the pedestrian has given the turn to the driver and expects them to take it.

There are big AI perception challenges, just in my neighborhood, to get driverless cars to interact with people as well us driverful cars do. What if level 4 and level 5 autonomy self driving cars are not able to make that leap of fitting in as equals as current cars do?

Cars will clearly have to be able to perceive people walking along the street, even and especially on a snowy day, and not hit them.  That is just not debatable.  What is debatable is whether the cars will need to still pass them, or whether they will slowly follow people not risking passing them as a human driver would.  That slows down the traffic for both the owner of the driverless car, and for any human drivers.  The human drivers may get very annoyed with being stuck behind driverless cars.  Driverless cars would then be a nuisance.

In the little side streets, when at a stop sign, cars will have to judge when someone is about to cross in front of them.  But sometimes people are just chatting at the corner, or it is a parent and child waiting for the school bus that pulls up right there.  How long should the driverless car wait?  And might someone bully such cars by teasing them that they are about to step off the curb–people don’t try that with human drivers as there will soon be repercussions, but driverless cars doing any percussioning will just not be acceptable.

Since there are no current ways that driverless cars can give social signals to people, beyond inching forward to indicate that they want to go, how will they indicate to a person that they have seen them and it safe to cross in front of the car at a stop sign?  Perhaps the cars will instead need to be 100% guaranteed to let people go.  Otherwise without social interactions it would be like the case of the dark country road.  In that case driverless cars would have a privileged position compared to cars with human drivers and pedestrians.  That is not going to endear them to the residents.  “These damn driverless cars act like they own the road!”  So instead, driverless cars will need to be very wimpy drivers, slowing down traffic for everybody.

At a cross walk in Central Square driverless cars potentially might be stuck for hours. Will people take pity on them as they do on human drivers? To take advantage of this the cars would need to understand human social signals of giving them a turn, but without a reciprocal signal it is going to be confusing to the generous pedestrians and they may soon decide to not bother being nice to driverless cars at all. That will only make it more frustrating for a human driver stuck behind them, and in Central Square at least, that will quickly lead to big traffic jams. “Damn those driverless cars, they just jam the place up!”

According to this report from the UK, there are predictions that traffic on highways will slow down somewhat because of timid autonomous systems until some threshold of autonomous density is reached.   I think the dynamics where we consider the role of pedestrians is going to be very different and much more serious.

If self driving cars are not playing by the unwritten rules of how pedestrians and other drivers expect cars to interact, there will be ire directed at someone.  In the case of cars with level 2 or level 3 autonomy there will be a driver in the driver’s seat, and pedestrians will see them, see their concerns being ignored by the person, and direct their ire at that person, most likely the owner or the person temporarily using the car as a service.  If the car is under level 4 or level 5 autonomy it may be totally unoccupied, or have no seating in what would be the driver’s seat, and then the ire will be directed at that class of car.

I see a real danger of contempt arising for cars with level 4 and level 5 autonomy.   It will come from pedestrians and human drivers in urban areas.  And when there is contempt and lack of respect, people will not be shy about expressing that contempt.

At least one manufacturer  is afraid that human drivers will bully self driving cars operating with level two autonomy, so they are taking care that in their level 3 real world trials the cars look identical to conventional models, so that other drivers will not cut them off and take advantage of the heightened safety levels that lead to autonomous vehicle driving more cautiously.

2. Anti-social Behavior of Owners

The flip side of autonomous cars not understanding social mores well enough, is owners of self driving cars using them as a shield to be anti-social themselves.

Up from Central Square towards Harvard Square is a stretch of Massachusetts Avenue that is mixed residential and commercial, with metered parking.  A few weeks ago I needed to stop at the UPS store there and ship a heavy package.  There were no free parking spots so I soon found myself cruising up and down along about a 100 meter stretch, waiting for one to open up.  The thought occurred to me that if I had had a level 4 or 5 self driving car I could have left it to do that circling, while I dropped into the store.

Such is the root of anti-social behavior. Convenience for the individual, me not having to find a parking spot, versus over exploitation of the commons, filling the active roadway with virtually parked cars. Without autonomous vehicles UPS locations that are in places without enough parking shed some of their business to locations that have more extensive parking. That dynamic of self balancing may change once car owners have an extra agent at their beck and call, the self driving system of their automobiles.

We have seen many groups, including Tesla, talk about the advantage to individuals  of having their cars autonomously dealing with parking, so from a technical point of view I think this capability is one that is being touted as an advantage of autonomous cars. However, it gets to interact with human nature and then anti-social behavior can arise.

I think there will be plenty of opportunity for people to take other little short cuts with their autonomous cars. I’m sure the owners will be more creative than I can be, but here are three additional examples.

(1) People will jump out of their car at a Starbucks to run in and pick up their order knowingly leaving it not in a legal parking spot, perhaps blocking others, but knowing that it will take care of getting out of the way if some other car needs to move or get by. That will be fine in the case there is no such need, but in the case of need it will slow everything down just a little. And perhaps the owner will be able to set the tolerance on how uncomfortable things have to get before the car moves. Expect to see lots of annoyed people. And before long grocery store parking lots, especially in a storm, will just be a sea of cars improperly parked waiting for their owners.

(2) This is one for the two (autonomous) car family. Suppose someone is going to an event in the evening and there is not much parking nearby. And suppose autonomous cars are now always prowling neighborhoods waiting for their owners to summon them, so it takes a while for any particular car to get through the traffic to the pick up location. Then the two car family may resort to a new trick so that they don’t have to wait quite so long as others for their cars to get to the front door pick up at the conclusion of the big social event. They send one of their cars earlier in the day to find the closest parking spot that it can, and it settles in for a long wait. They use their second car to drop them at the event and send it home immediately. When the event is over their first autonomous car is right there waiting for them–the cost to the commons was a parking spot occupied all day by one of their cars.

(3) In various suburban schools that my kids went to when they were young there was a pick up ritual, which I see being repeated today when I drive past a school at the right time. Mothers, mostly, would turn up in their cars just before dismissal time and line up in the order that they arrived with the line backing out beyond the school boundary often. When school was over the teachers would come outside with all the kids and the cars would pull up to the pick up point, the parents and teachers would cooperate to get the kids into their car seats, and off would go the cars with the kids, one at a time. When the first few families have fully driverless cars, one can imagine them sending their cars to wait in line first, so that their kids get picked up first and brought home. Not only does that mean that other parents would have to invest more of their personal time waiting in order to get their kids earlier, while the self driving car owners do not, but it ends up putting more responsibility on the teachers. Expect to see push back on this practice from the schools. But people will still try it.

Early on in the transition to driverless cars the 1% will have a whole new way to alienate the rest of the society. If you don’t think so, take a drive south from San Francisco on 101 in the morning and see the Teslas speeding down the left most lane.

What This Means

There are currently only fifteen fully driverless train systems in the United States, mostly in airports, and all with at most a handful of miles of track, all of which is completely spatially separated from any rights of way for any vehicles or pedestrians outside of the systems.  The first large scale driverless mass transit system in the US is going to be one that is under construction in Honolulu at this time, scheduled to be in initial operation in 2020 (though in late 2014 it was scheduled to begin operation in 2017).

There have been designs for larger scale systems to be driverless, for almost fifty years–for instance the San Francisco BART (Bay Area Rapid Transit) trains, first introduced in 1972 had lots of control automation features right at the beginning.  Failures and accidents however meant that many manual systems were added and sometimes later removed, sometimes having serious negative impact on overall efficiency of the system.

The aspirations for driverless train systems most closely correspond to level 4 autonomy for cars, but in very restrictive geographical environments.  Level 5 autonomy for trains would correspond to trains on tracks with level crossings, or street cars that share space with automobiles and pedestrians.  No one is advocating for, or testing, level 5 train autonomy at this moment.

Note also, that train navigation is very much simpler than automobile navigation.  There are guide rails!  They physically restrict were the trains can go.  And note further that all train systems are very much operated by organizations full of specialists.  Individual consumers do not go out and buy trains and use them personally–but that is what we are expecting will happen with individual consumers buying and using self driving cars.

Level 4 autonomy for trains is much easier than level 4 autonomy for cars.  Likewise for level 5.  But we hardly have any level 4 autonomous trains in the US.

Gill Pratt, CEO of Toyota Research Institute said just a few days ago that “none of us in the automobile or IT industries are close to achieving true Level 5 autonomy”.

The preceding two sections talked about two ways in which self driving cars are going to get a bad name for themselves, as social outcasts in situations where there are pedestrians and other drivers, and in enabling anti-social behavior on behalf of their owners. Even ignoring the long tail of technical problems remaining to be solved for level 5 autonomy, to which Pratt refers, I think we are going to see push back from the public against level 5 and against widespread level 4 autonomy.  This pushback is going to come during trials and early deployments.  It may well be fierce.  People are going to be agitated.

Technically we will be able to make reasonable systems with level 4 autonomy in the not too distant future, but the social issues will mean that the domains of freedom for level 4 autonomous vehicles will be rather restricted.

We’ll see autonomous trucks convoying behind a single human occupied truck (perhaps itself a level 3 vehicle) in designated lanes on highways. But once off the highway we’ll demand individual humans in each truck to supervise the off highway driving.

Just as in airports where we have had self driving trains for quite a while we’ll see limited geographic domains where we have level 4 autonomous cars operating in spaces where there are no pedestrians and no other human drivers.

For instance, it will not be too long before we’ll have garages where drivers drop off their cars which then go and park themselves with only inches on each side in tightly packed parking areas. Your car will take up much less room than a human parked car, so there will be an economic incentive to develop these parking garages.

Somewhat later we might see level 4 autonomy for ride hailing services in limited areas of major cities.  The ride will have to begin and terminate within a well defined geographic area where it is already the case that pedestrian and automobile traffic is well separated by fairly strong social norms about the use of  walk signals at the corner of every block.  Some areas of San Francisco might work for this.

We might also see level 4 autonomy on some delivery vehicles in dense urban environments.  But they will need to be ultra deferential to pedestrians, and not operate and clog things up for other cars during peak commuting periods.  This could happen on a case by case basis in not too many years, but I think it will be a long time before it gets close to being universally deployed as a means of delivery.

We’ll see a slower than techies expect deployment of level 4 autonomy, with a variety of different special cases leading the way.  Level 5 autonomy over large geographical areas is going to be a long time coming.  Eventually it will come, but not as quickly as many expect today.

The futurist Roy Amara was well known for saying: We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.

That is where we are today.  People are overestimating how quickly level 5 autonomy will come, and even over estimating how widespread level 4 autonomy will be any time soon.  They are seeing the technical possibilities and not seeing the resistance that will come with autonomous agents invading human spaces, be they too rude or overly polite. But things will march on and at some point every single car will be level 5 autonomy and we’ll no longer let people drive.  Eventually it will creep up on us and we’ll hardly notice when it does happen.

Eventually manual driving disappear in all but specialized entertainment zones.  But by then we won’t notice.  It is inevitable.  But, that day will not be soon.  And the flying cars will be even later.

---

---

And now we get to a little flaming:


<flame>


There is a serious question about how safe is safe.  35,000 people in the US are killed in motor vehicle accidents per year, with about 1.25 million world wide.  Right now all these deaths involve human drivers. They are both horribly large numbers.  Over the last 120 years we, the human race, has decided that such high numbers of deaths are acceptable for the usefulness that automobiles provide.

My guess is that we will never see close to such high numbers of deaths involving driverless cars.  We just will not find them acceptable, and instead we will delay adopting levels 4 and 5 autonomy, at the cost of more overall lives lost, rather than have autonomous driving systems cause many deaths at all.  Rather than 35,000 annual deaths in the US it will not be acceptable unless it is a relatively tiny number.  Ten deaths per year may be deemed too much, even though it could be viewed as minus 34,990 deaths.  A very significant improvement over the current state of affairs.

It won’t be rational. But that is how it is going to unfold.

Meanwhile, there has been a cottage industry of academics and journalists looking for click bait (remember, their whole business model got disrupted by the Internet–they are truly desperate, and have been driven a little mad), asking questions about whether we will trust our cars to make moral decisions when they are faced with horrible choices.

You can go here to a web site at M.I.T. to see the sorts of moral decisions people are saying that autonomous cars will need to make.  When the brakes suddenly fail should the car swerve to miss a bunch of babies in strollers and instead hit a gaggle of little old ladies?  Which group should the car decide to kill and which to save, and who is responsible for writing the code that makes these life and death decisions?

Here’s a question to ask yourself. How many times when you have been driving have you had to make a forced decision on which group of people to drive into and kill? You know, the five nuns or the single child? Or the ten robbers or the single little old lady? For every time that you have faced such decision, do you feel you made the right decision in the heat of the moment? Oh, you have never had to make that decision yourself? What about all your friends and relatives? Surely they have faced this issue?

And that is my point. This is a made up question that will have no practical impact on any automobile or person for the forseeable future. Just as these questions never come up for human drivers they won’t come up for self driving cars. It is pure mental masturbation dressed up as moral philosophy. You can set up web sites and argue about it all you want. None of that will have any practical impact, nor lead to any practical regulations about what can or can not go into automobiles. The problem is both non existant and irrelevant.

Nevertheless there is endless hand wringing and theorizing, in this case at Newsweek, about how this is an oh so important problem that must be answered before we entrust our cars to drive autonomously.

No it is not an important question, and it is not relevant. What is important is to make self driving cars as safe as possible. And handling the large tail of perceptual cases that arise in the real world will be key to that.

Over the years many people have asked me and others whether our robots are “three laws safe”. They are referring to Asimov’s three laws from his science fiction books in the 1950’s about humanoid robots.

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

But those who have actually read Asimov’s book know that Asimov used these laws as a source of plot, where ambiguities led to a plot twist, or where, through a clever set up, conflicts between the laws were introduced. They were a joke!  It has not stopped the press breathlessly picking up on this as an important factor for robots.  Almost as bad as how the press picks up on the Turing test (itself a rhetorical device used by Alan Turing to make a point, not an actual certification of intelligent behavior).  Not that it is all the fault of the press.  There are plenty of academics (and recently Lords, physicists, and billionaires) who have also chosen to draw attention to a supposed barrier to the use of AI–whether machines will be moral.  There is nothing sensible to say on these issues at this time.

For Asimov’s laws none of our robots or perception systems can figure out the state of the world well enough for any robot today, or in the forseeable future to figure out when which law applies. And we won’t have cars that can tell nuns from robbers–how about robbers dressed as nuns, all the better when out on a bank robbing spree?

The Newsweek article, somewhat tongue in cheek, suggests:
To handle these relative preferences, we could equip people with beacons on their cellphones to signal nearby cars that they are a certain type of person (child, elderly, pedestrian, cyclist). Then programmers could instruct their autonomous systems to make decisions based on priorities from surveys or experiments like the Moral Machine.
Err, yeah.  This is going to work well, as no robber is ever going to choose the nun setting on their phone–I’m sure they will identify themselves as a robber, as they should!

My favorite answer to this general moral dilemma, known as the trolley problem, was given by Nicholas, the two year old son of E. J. Masicampo who teachs a moral philosophy class. Seen here dad sets up Nicholas’ wooden train set so that taking one fork will kill one person, and the other fork will kill five. Asked what should the train do, Nicholas moves the singleton to lie on the same track as the other five, then drives his train into all six of them, scatters them all, and declares “oh, oh”!


</flame>


Footnotes

Interface Message Processors. Today they would be referred to as Internet protocol routers.

Bolt, Beranek and Newman in Cambridge, MA, a company that was always known as BBN. As distinct from BBN, the Buckingham Browne and Nichols school in Cambridge, MA — no doubt many employees of BBN sent their kids to school at BBN.

Like all things called “Squares” in Massachusetts there is absolutely nothing to do with squareness in Central Square.  It is just a region of Massachusetts Avenue in Cambridge where there is so much commercial activity that there are zero buildings with residential occupancy at ground level.

My kids all went to a private pre-school in Concord, MA, and almost all the parents owned dark blue Volvo 240DL station wagons.  Although our kids could all tell their parents’ car from the others at the grocery store, it just didn’t work at this pre-school.  The kids could never tell when it was their parent rolling up for the next pickup.  That was back when the 1% was a few more percentage points of the population, and not quite as hollowed out as now…

Full disclosure.  I am on the advisory board for the Toyota Research Institute, but this blog post represents only my own thoughts on autonomous driving.

Most people failed to notice that a technology, analog TV,  that had been omnipresent for most of their lives was overtaken and then one day it just disappeared as it did in the US on June 12, 2009.  Poof!  It was gone.