Autonomous Car Progress

[Soda Popinski]

Now this is a very good question. We have yet to see anyone demonstrate what a DDT fallback scenario looks like.

I had to Google DDT-Fallback, because what came to mind was this. I've been watching too much GLOW on Netflix.

 

[diplomat33]

If we've never seen a Waymo vehicle perform a DDT-fallback, how do we know it's level 4?

Because Waymo says in their guide to first responders that it is L4 and describes what the cars do during a fallback:

Excerpts from first responder manual:

The Waymo vehicle is fully self-driving.

• It is validated for driverless testing and operation.
• It is capable of performing the entire dynamic driving task within its operational design domain, as a Level 4 automated driving system under SAE International’s Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, standard J3016.
• It is capable of performing a safe stop, known as achieving a “minimal risk condition,” without any expectation that a human will need to intervene.
• It is equipped with redundancies for critical systems, such as sensors, computing, steering and braking, and can automatically detect changes to the vehicle or the environment and determine an appropriate response to keep the vehicle, its passengers, and other road users safe.

The Waymo vehicle uses its sensors to identify police or emergency vehicles by detecting their appearance, their sirens, and their emergency lights.

• The Waymo vehicle is designed to yield as appropriate to these emergency vehicles no matter which direction they are headed.

If a Waymo fully self-driving vehicle detects that a police or emergency vehicle is behind it and flashing its lights, the Waymo vehicle is designed to pull over and stop when it finds a safe place to do so.

• The vehicle can unlock the doors and roll down the windows for Waymo’s Rider Support team to communicate with law enforcement. Rider Support can also be reached by pressing the Help button in the interior console accessible from the second row passenger seating area.
• Waymo’s Rider Support specialists have protocols for interacting with any vehicle passengers in the event of the vehicle being pulled over or involved in a collision, by providing information through in-vehicle speakers, on the in-vehicle displays, and communicating with passengers through in-vehicle telecommunications capabilities.
• A Waymo support team will be dispatched to provide on-scene support, when needed, for passengers and first responders.

First Responders – Waymo

So I am not making it up that Waymo is L4. Waymo says that they are L4 in their official guide to first responder!

Are you saying the only thing stopping Smart Summon from being level 4 is a little more intelligence behind moving the car out of the right of way?

Not quite, to be L4, Smart Summon would need to be able to do all the dynamic driving tasks and fallback on its own in a parking lot ODD without a human having to hold down a button. For example, it would need to actually follow the direction of travel in a parking lot and not try to just cut across parking spaces. I am also not sure how good SS is at always stopping for hazards. I don't the car's cameras are able to always see small objects close to the car. Hence, SS is not monitoring the environment 100% like a L4 car should.

Doesn't it presently turn on the hazards, park itself, and summon assistance in the event of a failure?

No, it does not do that for me. SS just stops right there in the middle of the road. It's actually quite annoying to other vehicles and people in the parking lot because the car will just stop there, potentially blocking the path to other cars. So it is definitely not doing a good fall-back.

 

[diplomat33]

Stop trolling. A car that needs constant supervision is L2. Show me the plenty of evidence in the last 6 months? Oh you have none, wow. And tell me when you've ridden a driverless waymo. Oh I see you can't. Neither can 99.999% of the rest of the population. And the few that can ride don't get an unoccupied vehicle, that is reserved for the marketing stunts. You are really getting old.

The number of videos or how many people use the cars is irrelevant to the SAE levels.

Waymo says they are L4 in their First Responder guide on page 9:

The Waymo vehicle is fully self-driving.
● It is validated for driverless testing and operation.
● It is capable of performing the entire dynamic driving task within its operational design domain, as a Level 4 automated driving system under SAE International’s Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, standard J3016.
● It is capable of performing a safe stop, known as achieving a “minimal risk condition,” without any expectation that a human will need to intervene.
● It is equipped with redundancies for critical systems, such as sensors, computing, steering and braking, and can automatically detect changes to the vehicle or the environment and determine an appropriate response to keep the vehicle, its passengers, and other road users safe.

Source:
https://storage.googleapis.com/sdc-...forcement_interaction_protocol_2019-10-11.pdf

 

[daniel]

If the vehicle can perform the entire DDT and DDT-fallback on public roads in its ODD, even if the ODD is very limited (ie just an airport area), then it is L4.

Now since L4 has a limited ODD, it is certainly true that there can be a wide variation in L4 purposes. A roboshuttle in an airport and a robotaxi that is geofenced to the entire LA metro area, are both L4 even though their purposes are quite different. But both vehicles have the same characteristic of performing the entire DDT and DDT fallback in their ODD, hence they are both L4.

And this is precisely why I find the SAE level definitions useless for an ordinary person considering whether or not to buy a car. I'm not sure what purpose they do serve since they establish standards that have minimal relation to what the car can actually do in the real world. You could buy a car that's Level 4 and find out that it's only autonomous on a three-mile stretch of one county road in East Texas. That's a ridiculous extreme, but the point is that since L4 can have limited ODD, you don't know anything from the fact that the car is Level 4 unless you know the precise ODD.

As for consulting the dealership, as you suggested in an earlier post, the sales staff at dealerships are often completely ignorant of the capabilities of the cars they sell. When I bought my Prius, pretty much everything the salesman told me about the car was wrong. I still thought it was an excellent car, but the salesman didn't know jack spit about how it worked or what it would do. I eventually learned about the car from a chat board dedicated to that particular car.

When the Chevy Volt came out I was curious. I drove my Roadster to the Chevy dealership, told the salesman I was not in the market for a car but was interested in knowing about the Volt just out of curiosity. He let me drive the demo and we talked at length about the car. Much of what he told me was just plain wrong, as I learned later.

One of the great things about Tesla is that in TMC we have a place where we can find reliable information. When they release a new feature i can come here to learn what it actually does and whether it works as expected for everybody, nobody, or some number in between, with details. If Chevy came out with a car claimed to be Level 4 I don't know where I'd go for reliable information. Definitely not the dealership.

 

[FloridaJohn]

One problem I have is that the level 4 designation allows for geofencing. Thus a car that is capable of operating without a driver in a very narrow geographical area can be called L4, but it does me no good at all.

Just for the sake of the discussion, I'll also point out that the L4 ODD does not strictly have to be restricted to a geographic area. For example, the manufacturer could say that the car will drive autonomously except when there is snow on the ground. In that case, it would work in your environment without specifically calling out Maui.

 

[Daniel in SD]

If Chevy came out with a car claimed to be Level 4 I don't know where I'd go for reliable information.

Go to the website and they have a map of where Supercruise works (nowhere in Hawaii). It will probably be the same if they ever release an SAE Level 3 version.
The SAE standard is used as a piece of the regulatory framework. How vehicles are marketed to consumers isn't part of its scope (or the SAE's purpose).

 

[Soda Popinski]

If Chevy came out with a car claimed to be Level 4 I don't know where I'd go for reliable information. Definitely not the dealership.

I guess go to the GM Volt/Bolt equivalent of TMC, GM Volt Forum

 

[diplomat33]

And this is precisely why I find the SAE level definitions useless for an ordinary person considering whether or not to buy a car. I'm not sure what purpose they do serve since they establish standards that have minimal relation to what the car can actually do in the real world. You could buy a car that's Level 4 and find out that it's only autonomous on a three-mile stretch of one county road in East Texas. That's a ridiculous extreme, but the point is that since L4 can have limited ODD, you don't know anything from the fact that the car is Level 4 unless you know the precise ODD.

As for consulting the dealership, as you suggested in an earlier post, the sales staff at dealerships are often completely ignorant of the capabilities of the cars they sell. When I bought my Prius, pretty much everything the salesman told me about the car was wrong. I still thought it was an excellent car, but the salesman didn't know jack spit about how it worked or what it would do. I eventually learned about the car from a chat board dedicated to that particular car.

When the Chevy Volt came out I was curious. I drove my Roadster to the Chevy dealership, told the salesman I was not in the market for a car but was interested in knowing about the Volt just out of curiosity. He let me drive the demo and we talked at length about the car. Much of what he told me was just plain wrong, as I learned later.

One of the great things about Tesla is that in TMC we have a place where we can find reliable information. When they release a new feature i can come here to learn what it actually does and whether it works as expected for everybody, nobody, or some number in between, with details. If Chevy came out with a car claimed to be Level 4 I don't know where I'd go for reliable information. Definitely not the dealership.

You are right that the levels by themselves don't tell you everything. You need to know the ODD unless it is L5. But you will know the precise ODD. The automaker is required to tell customers what the ODD is. And the ODD must include everything from geofencing, time of day, max speed of the car, weather conditions, conditions that limit performance of FSD etc.. You will get all that information.

Unless you are a totally clueless customer who buys a car without doing any research (and that's the customer's fault), you will know the ODD. There will be lots of ways to find out the ODD. The car manual will tell you which you could probably find online before buying the car. The sales website for the car will probably tell you. GM has a map that shows you exactly where Supercruise works. I am sure there will be news websites, youtube videos, etc that will tell you. You can ask a friend who owns the car to tell you about the ODD. You are not limited to just asking an ignorant car salesman.

Obviously, none of us have any experience with buying a L4 car yet so it is natural to have concerns. You seem worried that you might buy a L4 car that does not work in Maui where you live. But don't worry. I don't think the ODD will be a problem. No, you are not going to buy a L4 car and suddenly find out that it only works on a 3 mile stretch of road in East Texas. Again, the ODD information will be available.

 

[daniel]

I not "worried that I might buy a L4 car that does not work in Maui where I live." I'm pointing out how useless the SAE designation is. The L4 designation is useless without a snot-ton of additional information. And if they're required to make that information public, I guarantee you that they will obfuscate it to the maximum extent that their lawyers can shoehorn into the requirements of the law. It's from chat boards like this one, not from the SAE designation or the owners' manual or the public pronouncements of the car maker that we'll find out what the car can actually do.

Of course some automakers are more trustworthy than others. I wouldn't trust GM as far as I could throw a Suburban.

 

[Daniel in SD]

I not "worried that I might buy a L4 car that does not work in Maui where I live." I'm pointing out how useless the SAE designation is. The L4 designation is useless without a snot-ton of additional information. And if they're required to make that information public, I guarantee you that they will obfuscate it to the maximum extent that their lawyers can shoehorn into the requirements of the law. It's from chat boards like this one, not from the SAE designation or the owners' manual or the public pronouncements of the car maker that we'll find out what the car can actually do.

Of course some automakers are more trustworthy than others. I wouldn't trust GM as far as I could throw a Suburban.

The map for where Super Cruise works is right on the main Super Cruise page!
I guess we'll see but I'm willing to bet that if Mercedes and Hyundai release SAE Level 3 systems next year like they say they will there will be maps of where they work prominently displayed on their websites.

 

[diplomat33]

I not "worried that I might buy a L4 car that does not work in Maui where I live." I'm pointing out how useless the SAE designation is. The L4 designation is useless without a snot-ton of additional information. And if they're required to make that information public, I guarantee you that they will obfuscate it to the maximum extent that their lawyers can shoehorn into the requirements of the law. It's from chat boards like this one, not from the SAE designation or the owners' manual or the public pronouncements of the car maker that we'll find out what the car can actually do.

I guess the SAE could have tried to incorporate some ODD information into the levels. But keep in mind that there are way too many parameters for ODD. ODD is not just geofencing. ODD also includes environmental, geographical, time-of-day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics. The different permutations of possible ODD are huge. It would be impossible to create a level for every single type of ODD without leaving something out. So it is better IMO to keep things more generic.

Here is what the SAE says:

Because of the wide range of possible ODDs, a wide range of possible features may exist in each level (e.g., level 4 includes parking, high-speed, low-speed, geo-fenced, etc.). For this reason, SAE J3016 provides less detail about the ODD attributes that may define a given feature than about the respective roles of a driving automation system and its user.

I think how the SAE chose to handle this issue makes sense. They chose to focus on the roles of the ADS and the user. I don't think that makes the levels useless. They still provide valuable information about the FSD car can do. For example, L3 tells you that the driver will need to take over at times. L4 tells you that you don't need a driver at all in some cases. L5 tells you that you don't need a driver ever. But it does mean that for levels other than L5, you will need to add extra information about the ODD if you want a truly complete picture of where and when the autonomous driving can operate.

 

[S4WRXTTCS]

The map for where Super Cruise works is right on the main Super Cruise page!
I guess we'll see but I'm willing to bet that if Mercedes and Hyundai release SAE Level 3 systems next year like they say they will there will be maps of where they work prominently displayed on their websites.

What's going to be interesting to see is how competitive they'll be. If they'll try to get a jump on the other company buy offering more roads or higher working speeds.

Or if the entire L3 will be artificially limited either due to regulations, liability. or imposed by the solution provider (like MobileEye).

So far no one has actually released an L3 vehicle. and it's always been just around the corner.

 

[Daniel in SD]

What's going to be interesting to see is how competitive they'll be. If they'll try to get a jump on the other company buy offering more roads or higher working speeds.

Or if the entire L3 will be artificially limited either due to regulations, liability. or imposed by the solution provider (like MobileEye).

So far no one has actually released an L3 vehicle. and it's always been just around the corner.

I also remain a skeptic. Maybe Hyundai has the best shot at being first since Korea has put in place rules for Level 3 vehicles.
In the US the regulations are different between states and all seem to be geared towards robotaxis where the liability issue is simple. We may have to wait for a major US automaker to lobby to get rules put in place for L3 systems.

 

[mikes_fsd]

The map for where Super Cruise works is right on the main Super Cruise page!

I love this little nugget.

 

[willow_hiller]

I love this little nugget.

And let me guess, it's a $300 surcharge for the dealership to flash the new firmware.

 

[mikes_fsd]

And let me guess, it's a $300 surcharge for the dealership to flash the new firmware.

Uhm, for maps on a Lexus they wanted to charge over $500 a few years back... (been a while) so expect to be charged something.

 

[daniel]

The map for where Super Cruise works is right on the main Super Cruise page!
I guess we'll see but I'm willing to bet that if Mercedes and Hyundai release SAE Level 3 systems next year like they say they will there will be maps of where they work prominently displayed on their websites.

And as diplomat33 points out in the following post, the map is just one very small part of the ODD. Automakers will obfuscate the rest of it to make it appear far more useful than it will actually be.

And I predict that when Tesla finally gets to Level 3 in five or ten years, it will be the first one to offer Level 3 that works here on Maui. (Which in a way is good because I really like my Tesla, but I give it a 50-50 chance that I won't last that long myself.)

I guess the SAE could have tried to incorporate some ODD information into the levels. But keep in mind that there are way too many parameters for ODD. ODD is not just geofencing. ODD also includes environmental, geographical, time-of-day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics. The different permutations of possible ODD are huge. <...snip...>

 

[diplomat33]

And as diplomat33 points out in the following post, the map is just one very small part of the ODD. Automakers will obfuscate the rest of it to make it appear far more useful than it will actually be.

But automakers will need to be careful though because if they are caught misleading the public, they risk legal penalties. And the ODD is something that will be very obvious once you drive the car. So it will be super easy to prove in court if the automaker clearly lied about the ODD.

And I predict that when Tesla finally gets to Level 3 in five or ten years, it will be the first one to offer Level 3 that works here on Maui. (Which in a way is good because I really like my Tesla, but I give it a 50-50 chance that I won't last that long myself.)

If it takes Tesla 5-10 years to get to L3, then I predict someone else will provide L3 in Maui first before Tesla does. Consider that many automakers are aiming to provide L3 in about 3 years.

 

[diplomat33]

"Not all automated technology can be treated as equal. As partial, semi and fully autonomous vehicle technologies become more prevalent on public roads, it’s critical that companies, members of the media and the public avoid “autonowashing” technology with disparate and specific uses. Liza Dixon, an engineer and researcher in the field of human-centered automation, shares why we should care about autonowashing, what needs to be done to avoid it, and how we can make our roads safer through education."
Let's Talk Self-Driving Live Events | Let's Talk Self-Driving

 

[daniel]

But automakers will need to be careful though because if they are caught misleading the public, they risk legal penalties. And the ODD is something that will be very obvious once you drive the car. So it will be super easy to prove in court if the automaker clearly lied about the ODD.

There are laws against fraud. There is no law against "misleading" the public. The entire science of advertising is how to mislead without breaking the law. And lawyers have made a science of how to seem to be saying something you are not. They can make a true statement appear to mean something entirely different. The ODD will be accurately described in five hundred pages of jargon so arcane that nobody can understand it, but which seems to the layperson to be describing our common-sense notion of "It will work anywhere a human would normally be willing to drive."

And even if they were caught lying and fined, when was the last time a big corporation had to pay fines more than about 1% of the profit they made with their lies? Never. The tobacco companies were proven to have knowingly lied about the deadly consequences of smoking, were proven to have known all along that millions of people would die of smoking, and lied about it, and they got fined an insignificant percentage of the profits they made from those lies, and the government even applied pressure on foreign countries to stop them from opposing smoking.

Don't count on the government to enforce any kind of openness on corporations.