Autonomous Car Progress

[stopcrazypp]

I posted about this here: California Autonomous Vehicle Collision Reports
What doesn't make sense to me is that it sounds like the Prius would have had the right of way if they were turning right but the Cruise vehicle turned in front of them. Perhaps it was predicting that the Prius would slow down to make a right turn? Also, once detecting that the possibility of a collision the Cruise vehicle stopped? It sounds like without that inhuman behavior there wouldn't have been a collision.
Looking forward to seeing the report (in a year or two probably, haha).

I responded to your post a bit more in the other thread:
California Autonomous Vehicle Collision Reports

This is probably something a local driver would have avoided given they would know some people violate the bus/turn only lanes, but probably the Cruise vehicle did not expect.

 

[hgmichna]

As to deaths per autopilot miles, has anybody determined how many deaths were caused by autopilot? You can't blame autopilot when somebody jumps on the moving car.

 

[Dewg]

As to deaths per autopilot miles, has anybody determined how many deaths were caused by autopilot? You can't blame autopilot when somebody jumps on the moving car.

There is a well researched site called tesladeaths that shows claimed and verified deaths.

 

[thewishmaster]

More info on Cruises getting stuck in the middle of the street: Cruise’s Robot Car Outages Are Jamming Up San Francisco

I guess that answers the question of whether they can function offline (or at least whether Cruise allows them to do so)

 

[diplomat33]

More info on Cruises getting stuck in the middle of the street: Cruise’s Robot Car Outages Are Jamming Up San Francisco

I guess that answers the question of whether they can function offline (or at least whether Cruise allows them to do so)

Thanks for the information. Hopefully it is something they are able to solve soon.

 

[Daniel in SD]

More info on Cruises getting stuck in the middle of the street: Cruise’s Robot Car Outages Are Jamming Up San Francisco

I guess that answers the question of whether they can function offline (or at least whether Cruise allows them to do so)

Wow, doesn't sound great. Still doesn't explain why they end up in clusters. Interesting that they've cut back on unprotected lefts after the Prius collision. Disappointing that these systems still seem to have so many problems.

 

[AlanSubie4Life]

Disappointing that these systems still seem to have so many problems.

This is where FSD will shine, since it does not rely on an internet connection and does not use HD maps. The first year, progress may have seemed slow, but soon they will hit the singularity.

 

[diplomat33]

Disappointing that these systems still seem to have so many problems.

Yes it is disappointing. But teaching a robot to drive on public roads with other humans, ie autonomous driving, is super hard.

My understanding of the current state of autonomous driving is that perception is good enough and prediction/planning is mostly good enough. That is why AVs like Waymo and Cruise can do safe driverless in easy urban settings. The last big remaining challenge is "solving" prediction/planning because AVs have to deal with humans who drive erratically. IMO, if human drivers always followed the rules, AVs would already be good enough now for mass deployment. But since humans don't always drive by the rules, it is hard for AVs to anticipate what other road agents will do. If you look at the accident with the Prius, it's the Prius that caused the accident by driving in the wrong lane and speeding. If the Prius had obeyed the road rules, the accident would not have happened. The Prius accident is a perfect example of what AVs still need to solve. They need to be able handle those cases where human drivers don't follow the rules. Of course, prediction will never be 100%. But AVs need to be as good as possible at anticipating when other road agents drive erratically. It is what a good defensive human driver is able to do. So IMO, the AV that is good enough at driving defensively and anticipating other roads agents will be safe enough to deploy at scale.

What is interesting to me is that there are two camps about how to "solve" the prediction/planning problem I mentioned above.

You have the Waymo camp which is trying to solve it with lots of clever ML. So Waymo has come up with new ML techniques for predicting paths and intent of objects. One ML model predicts paths of a single object very accurately but takes a lot of compute power to scale to lots of objects. Then there is another ML model that can approximate for lots of objects with low compute power but is less accurate. So Waymo has come up with a way to take information from the first model to train the second model in order to get the best of both worlds: a ML model that is accurate with lots of objects and uses low compute power.

Then you have the Mobileye camp which believes that trying to predict paths for lots of objects takes too much compute power. So Mobileye is not trying to predict the paths of every object. Instead, Mobileye's approach is to essentially create rules for defensive driving. The idea is that the Mobileye AV might not perfectly predict the paths of other objects but it will be safe enough because it will follow good driving principles like maintaining a safe envelope, yielding to other vehicles etc...

 

[Supcom]

Worst of all, the company was unable to access its fallback system, which allows remote operators to safely steer stopped vehicles to the side of the road.

What?? The safety driver does not have ultimate control over the vehicle? I suspect that, with loss of a connection to the Cruise Command and Control Center, the car assumed that its signal was being jammed in a theft attempt and locked out the accelerator so that the safety driver couldn't steal the car.

Curious that the cars apparently are not programmed to autonomously drive to a safe parking spot when comms to the mothership are severed. Maybe cruise robotaxis aren't quite as autonomous as they should be.

 

[Daniel in SD]

What?? The safety driver does not have ultimate control over the vehicle? I suspect that, with loss of a connection to the Cruise Command and Control Center, the car assumed that its signal was being jammed in a theft attempt and locked out the accelerator so that the safety driver couldn't steal the car.

Curious that the cars apparently are not programmed to autonomously drive to a safe parking spot when comms to the mothership are severed. Maybe cruise robotaxis aren't quite as autonomous as they should be.

Cruise is operating without safety drivers (which is of course the point of robotaxis!).
I think they don't want to risk not knowing where their cars are or what they're doing. They probably hoped their connection to their vehicles would be reliable enough for this to be a very rare event. Seems to me that they should go back to having safety drivers until they resolve this issue.

 

[Dewg]

I think there will never be 100% flawless AV driving as long as humans are involved. Doesn't matter how safe and predictive AVs or Tesla FSD are, an idiot/drunk driver will still slam into them or cause them to panic and possibly chain react other accidents. The goal is to make them safer than humans.

 

[Dan D.]

Cruise is operating without safety drivers (which is of course the point of robotaxis!).
I think they don't want to risk not knowing where their cars are or what they're doing. They probably hoped their connection to their vehicles would be reliable enough for this to be a very rare event. Seems to me that they should go back to having safety drivers until they resolve this issue.

Some good information here about interacting with Cruise in emergency situations

In my opinion, if they need people to call Cruise directly at 888-662-7103 to deal with emergency situations, they should put this number ON THE CAR. It's all well for emergency responders to be trained, but what about the 2-3 minutes before they get on scene, what are regular people to do?
My Suggestion: have a kill switch on the outside sides of the car, or at the very least a panic button like on trains. Yes, it's nice that there are ways INSIDE the car, but there needs to be some OUTSIDE too.

Scenario: the door handle/bumper somehow catches to the leash of a pet or handle of a stroller. The car starts to move away. What to do?
My Suggestion: smash a window, kick the car hard, yell "stop", stand in front of the car (if you dare). The car supposedly has crash sensors and microphones but I have no idea how sensitive they are. Probably the car will notice if you smash a window. Maybe.
Also, jam a large object or two under the tires as seen in the video

They talk in the video about how Cruise has trained operators standing by, yada yada yada. Obviously this doesn't work when the communication system breaks down. A stop the car NOW! button is needed. There should also be a clear "This Car Is In Manual Mode and is Safely Stopped" light - like a taxi light.

Probably in time, these kinds of things will be sorted out. Or they could just be proactive...

 

[diplomat33]

Some good information here about interacting with Cruise in emergency situations

In my opinion, if they need people to call Cruise directly at 888-662-7103 to deal with emergency situations, they should put this number ON THE CAR. It's all well for emergency responders to be trained, but what about the 2-3 minutes before they get on scene, what are regular people to do?
My Suggestion: have a kill switch on the outside sides of the car, or at the very least a panic button like on trains. Yes, it's nice that there are ways INSIDE the car, but there needs to be some OUTSIDE too.

Scenario: the door handle/bumper somehow catches to the leash of a pet. The car starts to move away. What to do?
My Suggestion: smash a window, kick the car hard, yell "stop", stand in front of the car (if you dare). The car supposedly has crash sensors and microphones but I have no idea how sensitive they are. Probably the car will notice if you smash a window. Maybe.
Also, jam a large object or two under the tires as seen in the video

They talk in the video about how Cruise has trained operators standing by, yada yada yada. Obviously this doesn't work when the communication system breaks down. A stop the car NOW! button is needed.

Probably in time, these kinds of things will be sorted out. Or they could just be proactive...

But wouldn't an outside kill switch allow people to mess with the car? I could see a situation where a Cruise AV is stopped at a red light or stop sign and some punks decide to play a prank by approaching the Cruise and hitting the kill switch to cause the Cruise to freeze in the middle of the road.

 

[Dewg]

My Suggestion: have a kill switch on the outside sides of the car, or at the very least a panic button like on trains. Yes, it's nice that there are ways INSIDE the car, but there needs to be some OUTSIDE too.

In an ideal society that's a great solution. But with humanity sliding down the drain, loaded with a-holes, that will be abused over and over for kicks/revenge/TikTok trends.

 

[Dan D.]

But wouldn't an outside kill switch allow people to mess with the car? I could see a situation where a Cruise AV is stopped at a red light or stop sign and some punks decide to play a prank by approaching the Cruise and hitting the kill switch to cause the Cruise to freeze in the middle of the road.

In an ideal society that's a great solution. But with humanity sliding down the drain, loaded with a-holes, that will be abused over and over for kicks/revenge/TikTok trends.

Yes, people are jerks. Put a safety cover on the switch. How many people press the panic button on a train or open an emergency door on a plane? Some, and they are punished with fines. How long do stupid TikTok trends continue? A few stories and then people ending up in court (or the hospital) and the trend stops. People will be stupid, be publicly shamed, learn, and hopefully stop doing it.

What is your suggestion to immediately stopping a wayward autonomous car that has caused damage/injury or is about to do so? I'm open to other suggestions? I doubt calling the 888-662-7103 number - once you've actually found it on Google - and giving the operator the car ID, location, situation, and dealt with whatever system they have to deter abuse is going to take any less than 2 minutes. Hardly an immediate way of stopping the car. Button pushed, car stops. You're probably on video if they find it happens as a prank - and it will happen as a prank, we'll have to deal with that.

 

[Dan D.]

Surely these won't be the last:

Man steals AV
Messing with AV's
Messing with AV's, article

 

[Daniel in SD]

Some good information here about interacting with Cruise in emergency situations

In my opinion, if they need people to call Cruise directly at 888-662-7103 to deal with emergency situations, they should put this number ON THE CAR. It's all well for emergency responders to be trained, but what about the 2-3 minutes before they get on scene, what are regular people to do?
My Suggestion: have a kill switch on the outside sides of the car, or at the very least a panic button like on trains. Yes, it's nice that there are ways INSIDE the car, but there needs to be some OUTSIDE too.

Scenario: the door handle/bumper somehow catches to the leash of a pet or handle of a stroller. The car starts to move away. What to do?
My Suggestion: smash a window, kick the car hard, yell "stop", stand in front of the car (if you dare). The car supposedly has crash sensors and microphones but I have no idea how sensitive they are. Probably the car will notice if you smash a window. Maybe.
Also, jam a large object or two under the tires as seen in the video

They talk in the video about how Cruise has trained operators standing by, yada yada yada. Obviously this doesn't work when the communication system breaks down. A stop the car NOW! button is needed. There should also be a clear "This Car Is In Manual Mode and is Safely Stopped" light - like a taxi light.

Probably in time, these kinds of things will be sorted out. Or they could just be proactive...

I don't think that would do much to improve safety. I bet simply standing next to the car will cause it to stop and ask for remote assistance.
Having people lunging towards a moving car to press a button sounds like it would cause more safety problems than it would solve.

 

[Supcom]

How many people press the panic button on a train or open an emergency door on a plane?

Sadly, I regularly read reports of drunk/troupbled individuals attempting to open a passenger aircraft door in flight.

And I have personally seen individuals use the emergency door releases on light rail trains when they were too impatient to wait for the train operator to unlock them. Urban environments are tough places.

 

[Dan D.]

I don't think that would do much to improve safety. I bet simply standing next to the car will cause it to stop and ask for remote assistance.
Having people lunging towards a moving car to press a button sounds like it would cause more safety problems than it would solve.

We'll see. If the cars are programmed to navigate out of a stadium parking lot or busy city street and be "aggressive", they are going to have to deal with people being right next to them without stopping. How do you determine whether someone is standing near the car or there is a person trying to stop the car? Touch? Should a car stop immediately when someone is next to the car and touches it? That will cause the same gridlock. Safety button behind a mollyguard provides clear evidence of a problem, and willful abuse of the system is a crime.

If I'm driving through crowds while leaving the fireworks parking lot there are people everywhere. Sometimes they bump your car while you're stopped and they are crossing just because there's lots of people around. That's fine. Sometimes you tap the side of a taxi to say - ok off you go. Should that car stop? No. These are typical reactions of pedestrians around cars.

How do you stop a car that has run over someone, well it's pretty hard to do, usually they crash or they get away. You might see people smash the window and try to pull out the driver. How would you stop an AV that has run over someone and didn't seem to detect it, or was about to run over someone? There has to be a way or we will not allow AV's to continue operating.

 

[Daniel in SD]

We'll see. If the cars are programmed to navigate out of a stadium parking lot or busy city street and be "aggressive", they are going to have to deal with people being right next to them without stopping. How do you determine whether someone is standing near the car from the person trying to stop the car? Touch? Should a car stop immediately when someone is next to the car and touches it? That will cause the same gridlock. Safety button behind a mollyguard provides clear evidence of a problem, and willful abuse of the system is a crime.

I just think there's zero chance a robotaxi company will take the risk of running over someone's toes. I've never had to drive with people that close to my car (though I suppose I might have not noticed because they were outside of my view!). Robotaxis should absolutely stop if someone touches the car! The problem with the safety button is that it's only safe to press while the car is already stopped and if it's already stopped how do you know it's malfunctioning? Seems like it would be a very rare case.