I was going to write this post later this week filling in my promised experience from Thursday Oct 19th, 2023, experience of real fear that I might be involved in a really bad accident, while riding in a Cruise driverless taxi in San Francisco. The reason for rushing it out today is this story, today, that Cruise is no longer allowed to operate driverless taxis in San Francisco.
The story implies that they will no longer be allowed to operate even empty vehicles without a safety driver, which would mean two steps backwards from where they have been in San Francisco. It also says that Cruise misrepresented safety information to regulators.
My Recent Experience
I have taken around 36 Cruise driverless taxi rides over the last handful of months. They have had their ups and downs in user experience, and I had planned to talk about some of those in explaining why I do not think the experience is really what one expects from a deployed taxi service like Uber or Lyft.
But last Thursday night I had a moment where I experienced real fear, where for half a second I thought I might be involved in an extremely bad accident.
It was at night and we were crossing Divisadero, heading west, on Filbert. Left is a steep uphill few blocks on Divisadero. There was a car coming down the hill quite fast, as we crossed Divisadero. My Cruise, with nothing at all in front of it, braked hard, really hard, right in the middle of the intersection, harder than I had ever experienced a Cruise taxi braking. That brought us (me and my taxi) to almost a complete stop right in the path of the oncoming vehicle. Fortunately the other vehicle started to slow down and then the Cruise moved on out of its way.
This, above, is my recollection of what happened. When it braked hard a real pang of fear shot through my body. When I saw the car heading right at us a conscious version of that fear kicked in.
A human driver in that situation would mostly likely continue to drive and not brake at all. Braking was the best possible way to cause a collision. Not a good choice.
In previous accidents that have resulted in collisions Cruise vehicles have been at a stop. My interpretation, and I have no knowledge of whether this is true or not, was that rather than take the risk of hitting another vehicle while moving, the algorithms were set to freeze when there was an imminent collision, as better than running into someone else. A weird hard-wired trolley problem solution which does not protect the Cruise vehicle, but unfortunately for a rider does not protect them either. And in many cases increases the likelihood of a collision rather than reduces it.
See a Cruise with a passenger freezing in the middle of an intersection back in August, getting hit and sending a passenger to hospital.
More to come…
Your guess at what causes Cruise vehicles to stop certainly makes sense to me. After all, if the s/w finds itself in a situation that it doesn’t understand, stopping ASAP sounds like a good idea. There’s also perhaps a calculation on the part of the designers that their car is less likely to be blamed as the cause of the accident if it’s stationary at the moment of contact.
The technology may well reach a point where it can be proven that it causes fewer accidents than human drivers, but the accidents are likely to be of a different nature. I suspect people will not accept this kind of parity.
I don’t have nearly the years of experience in robotics that you have, or even as much experience as some of the Cruise engineers. But in 17 years of researching, designing and building control systems I’ve learned that the real world is humbling. My point is that even I could see what a big risk it is to let robo-taxis operate without a safety driver. How is it that more seasoned experts justify these risks to others? And may I be bold and ask , have all 36 rides been fully autonomous (without safety driver)?
The three rides I took in May 2022, and the three dozen I took over the last few months (both back when I was in a special “free” program, and since then when I have been a paying passenger) were all with no human, other than me and friends, in the car, and no remote human in the driving loop.
The first year AI terrified me was 2015. Malfunctioning automatic windshield wipers nearly killed me & 2 other people (a driver and Vivek Wadwha) in a rain storm at high speeds at 4am in France. https://joanna-bryson.blogspot.com/2016/01/2015-first-year-ai-nearly-killed-me.html
Rodney, you had good reason to worry. Cruise’s whole safety program is simply “stop immediately”. But that just isn’t safe. A couple of days ago a motorcyclist was injured because they ran into a suddenly stopped Cruise. But the scariest example is where a Cruise stopped in the middle of an intersection because a car was coming quickly toward them, just like your situation. The resulting crash was spectacular. Fortunately the Cruise was empty at the time. See video at https://twitter.com/miss_elenius/status/1692733759823446131
And note the details of the software update done just before the crash. It clearly did not work! https://x.com/skibuSmith/status/1692774297364734182?s=20
And a large detailed list of AV mishaps can be seen at https://www.safestreetrebel.com/av-failures/ . Safety has not been Cruise’s forte.