Elon: "Feature complete for full self driving this year"

[Bernesto]

Sorry, I should have probably addressed the 'left' turn portion of your comment. Not going to lie, that one makes me nervous still... Crossing traffic is part art, part balls, and totally the most dangerous part of driving. But, playing devil's advocate, the car has radar, and potentially can see more than we can because of that.

 

[Jeff Hudson]

This is quite the interesting conversation. If I might chime in, it seems that Tesla has a bit of a quandary about how to roll out FSD programming improvements as they approach actual government permitted FSD. What I mean is a driver can easily can become complacent with the ability of their vehicle to perform near FSD maneuvers and lose sight of the need to properly supervise the driving behavior. As we all know there are many different people who currently use Autopilot and some of us are much more observant of the Tesla warnings and actually obey them while at the opposite end of the bell curve others claim proudly their vehicle drives itself and even have videos of themselves driving without holding the steering wheel. It makes me wonder if they have actually read the Owner's Manual. It is these drivers that Tesla needs to somehow convince to take Autopilot more seriously as the capabilities grow into a real Level 3 for all roads but not yet up to government/insurance expectations to declare a legal FSD Level 4 with 99.999% reliability. I expect you would agree with me that Level 5 Autonomy is 99.9999% reliability. Tesla needs to either use the interior camera to ascertain the level of driver awareness or in the absence of the interior camera use more frequent driver interactions with the Autopilot software in control to allow continued Autopilot driving. Like I said this is going to get tricky for Tesla or any automaker for that matter as they roll out automation software to drivers who may not have a very good understanding of what they are using.

By the way Bernesto I prefer the Autonomous Driving description provided here on this SAE blog. The description is provided by ANSI.
SAE Levels of Driving Automation - ANSI Blog
I think this excerpt best illustrates why I like this description so much.
"These include the dynamic driving task (DDT), or all the real-time operational and tactile functions required to operate a vehicle in on-road traffic. Next is operational design domain (ODD), or the specific conditions under which a given driving automation system or feature thereof is designed to function. And then there’s the automated driving system (ADS), which encapsulates all the hardware and software collectively capable of performing the entire DDT on a sustained basis."
These DDT, ODD and ADS additions to any Autonomous Driving discussion serve to clear up confusion about what is meant about the various Automated Driving Levels.

 

[daniel]

Ofcourse, you start with zero trust

There are always people who want to be the first to try something out. I'm not one of them.

I disagree. All of these issues will be less dangerous for a car to negotiate than for a human driver. The car has the advantage of accessing precise GPS data, a cross referenced dataset of known intersections, and then mix that with a billion mile dataset of prior driver behaviors. Something that a human driver can not do...

I propose that people overestimate their capabilities, fear of lack of control, and a system 'untested by them' vs. it's real capabilities.

That said, I had a discussion with a colleague over lunch about this very thing. And, I personally don't believe Elon will roll this out until they hit six 9's of reliability. The cost of being wrong is just to high. Not just for Tesla, but for autonomous cars in general... Queue Good Morning America, "Killer Teslas" - Done, over...

Will FSD it be perfect, and never fail? No. Imposible. Will people die with vehicles in control? Yes. It just needs to be safer that the median driver on the road to be a success... But even then, people are still scared to fly, and we all know it's the safest form of travel!

I agree with all of the above once the system is mature. But every system has bugs when it's first released, and I will not be the first to try out NoA/city. (I wouldn't be the first anyway because I didn't pay for FSD, so if I do upgrade, I won't have it until it's been in operation for a year or two. But my point is that I'd be afraid to try it until other people have used it for a little while first.) When I first got my car it took me two weeks to get up the courage to turn on autosteer. Now I use it all the time.

The other thing is that when NoA/city is released, it will be Level 2, meaning the driver's full attention is required. The car will make the maneuvers, but the driver is still considered responsible for safety, and there's a lot more to be aware of when the car is making an unprotected left-hand turn than when it's just driving in its lane and adjusting its speed.

Once Level 5 no-driver-required has been on the roads in numbers for a month, I'll be ready to buy one. But I want to read those first-month reviews here first.

 

[Bernesto]

By the way Bernesto I prefer the Autonomous Driving description provided here on this SAE blog. The description is provided by ANSI.

Yes, this is a better and less ambiguous description of the levels. I specifically like how they call out the various functions and refer to the task vs the car...

The difficulty in this whole topic of FSD is the performance level of each ODD system from individual auto manufacturers. On paper my Acura has the same driver assist features as the Tesla. But in reality, on the Acura they are terribly unreliable, and a gimmick at best. These features are the predecessors to FSD ODD systems, and should have tests that each must pass prior to being let loose on the road.

Which brings up an interesting point... Is there even a entity and detailed criteria that exists by which each manufacturer will gain some sort of certification and scoring in their vehicle's ability to manage each DDT and ODD by their ADS, or are we just leaving that to manufacturers and just supposed to take their word for it? And that really goes for any driver assist feature on the road today! It seems to be that everything from 'lane keep' and 'traffic aware cruise' are moving drivers away from the task, and should have a series of government tests they have to pass constantly, just like airbags and antilock brakes prior to being allowed on the road...

While my car does a phenomenal job psudo-self driving on highways, I never let my guard down with it... Not for a second. I don't mind living on the bleeding edge... But, I understand FSD is wild west, and not ready for prime time... Lowest common denominator driver has to be addressed before FSD is released even in beta form, as 'that dirver' will let the car drive right into a truck while watching netflix...

"Look mom... No hands" crash "Look mom... no brains"....

 

[Bernesto]

I agree with all of the above once the system is mature. But every system has bugs when it's first released, and I will not be the first to try out NoA/city. (I wouldn't be the first anyway because I didn't pay for FSD, so if I do upgrade, I won't have it until it's been in operation for a year or two. But my point is that I'd be afraid to try it until other people have used it for a little while first.) When I first got my car it took me two weeks to get up the courage to turn on autosteer. Now I use it all the time.

The other thing is that when NoA/city is released, it will be Level 2, meaning the driver's full attention is required. The car will make the maneuvers, but the driver is still considered responsible for safety, and there's a lot more to be aware of when the car is making an unprotected left-hand turn than when it's just driving in its lane and adjusting its speed.

Once Level 5 no-driver-required has been on the roads in numbers for a month, I'll be ready to buy one. But I want to read those first-month reviews here first.

This is a very safe/smart position to take. And, to be honest, even if I had a L5 car, I'd still supervise the damn thing, all software has bugs.

I 'do' have the latest FSD on mine, and it to me is more of a 'like to try the latest thing" motivation.

In your case, I think even more important than reading the reviews, and letting it mature before you get one, is keeping an eye out for the guy driving down your street that already has one...!

 

[daniel]

Which brings up an interesting point... Is there even a entity and detailed criteria that exists by which each manufacturer will gain some sort of certification and scoring in their vehicle's ability to manage each DDT and ODD by their ADS, or are we just leaving that to manufacturers and just supposed to take their word for it?

That's why we have independent product testers like Consumer Reports. FSD cars will have to get authorization from regulators, and in the U.S. that will probably be on a state level. But CU and all the different car magazines will be testing these things as soon as they are available.

... I think even more important than reading the reviews, and letting it mature before you get one, is keeping an eye out for the guy driving down your street that already has one...!

This just falls under the category of normal defensive driving. You already need to watch out for bad drivers. The worst FSD will be better than some drivers. Cars are dangerous and people are idiots. The surprising thing is that there's anybody left at all. (Though the way we're going it's anybody's guess how long that will last.)

 

[Jeff Hudson]

You bring up some real good points Bernesto. This reminds me of one of my favorite topics concerning the physical and mental actions of driving a vehicle which is that they are mainly based on assumptions about what is about to happen based on what is observed. I wonder how low the percentage of drivers we share the road with understand any of what we are discussing concerning autonomous driving?

 

[Bernesto]

You already need to watch out for bad drivers. The worst FSD will be better than some drivers. Cars are dangerous and people are idiots. The surprising thing is that there's anybody left at all. (Though the way we're going it's anybody's guess how long that will last.)

LMAO. So true!! 1+

 

[Daniel in SD]

Which brings up an interesting point... Is there even a entity and detailed criteria that exists by which each manufacturer will gain some sort of certification and scoring in their vehicle's ability to manage each DDT and ODD by their ADS, or are we just leaving that to manufacturers and just supposed to take their word for it?

Basically we are relying on manufacturers to be responsible. California requires them to have a $5 million dollar liability insurance policy. And of course all accidents and disengagements must be reported to the DMV. I think realistically there is no way to test autonomous vehicles except in the real world. Determining how safe they are is a problem best left to statisticians.

 

[Bernesto]

I wonder how low the percentage of drivers we share the road with understand any of what we are discussing concerning autonomous driving?

This is true, purley by engaging in this conversation, we are more tuned into the thought process of 'driving' than 99% of people on the road.

Further, I often wonder, of the millions of vehicles sold each year with new driver assist options like Lane Keep, and TACC; what percentage of those vehicle owners actually use or understand those features, let alone relate to the new FSD future soon sharing the roadways with them...

Them: "Ooo, I like the red one... Does it have blue teeth? The kids need their own or they'll fight..." ‍

 

[Bernesto]

Basically we are relying on manufacturers to be responsible. California requires them to have a $5 million dollar liability insurance policy. And of course all accidents and disengagements must be reported to the DMV. I think realistically there is no way to test autonomous vehicles except in the real world. Determining how safe they are is a problem best left to statisticians.

Yeah, I figured as much. It's crazy, the feds require a trunk release inside the trunk in case you're kidnapped, but, standards for FSD cars, naw... lol

I know government moves too slow and would impede progress, but some basic standards committee staffed by leading manufacturers that design some basic tests for ODD systems to pass would be a step in the right direction.

And yes, the real world will be the final testing ground...

 

[Jeff Hudson]

understand those features, let alone relate to the new FSD future soon sharing the roadways with them

After following a few threads in these Tesla Forums for the past few years and using Tesla owners comments and personal observations as a guide to who among us have actually read the Owner's Manual I would venture a damn small percentage of all drivers really understand what is happening. This train of thought is beginning to depress me. Yeah, Tesla with their world class driving automation system is going to be hard pressed to roll out the expected improvements without owner/driver missteps setting their efforts back.

I agree with you about government regulation needing to take a more leading role and yet they need to do it without stifling innovation.

 

[Bernesto]

who among us have actually read the Owner's Manual

Well, it would help if the digital owner's manual was a bit easier to navigate... Or implemented a search function... Or if the index actually went to what it was linked to. Yes, I'm very familiar with it, and can see why no one else would have that level of patience. Well, other then you. LMAO

 

[Daniel in SD]

Yeah, I figured as much. It's crazy, the feds require a trunk release inside the trunk in case you're kidnapped, but, standards for FSD cars, naw... lol

I know government moves too slow and would impede progress, but some basic standards committee staffed by leading manufacturers that design some basic tests for ODD systems to pass would be a step in the right direction.

And yes, the real world will be the final testing ground...

It's because no one knows how to "test" self driving except in the real world. Current autonomous vehicles could easily pass contrived tests. Autonomous vehicles passed a test during the DARPA Urban Challenge in 2009. Keep in mind that the accident rate for humans is about once every 150,000 miles. Manufacturers have built test courses for AVs but realistically there are only so many permutations of tests that they can create and I'm sure they pass them all.

 

[Jeff Hudson]

Speaking of the Tesla Owner's Manual I wonder how many people have read the section on windshield wipers. Yeah, what do those settings really mean? | || ||| |||| Auto
| ------ means slow intermittent
|| ------ means fast intermittent
||| ----- means slow continuous
|||| ---- means fast continuous
AUTO means AUTO

 

[Bernesto]

It's because no one knows how to "test" self driving except in the real world. Current autonomous vehicles could easily pass contrived tests. Autonomous vehicles passed a test during the DARPA Urban Challenge in 2009. Keep in mind that the accident rate for humans is about once every 150,000 miles. Manufacturers have built test courses for AVs but realistically there are only so many permutations of tests that they can create and I'm sure they pass them all.

I partially agree. You are right, you can not contrive a test for every situation. But, I'd bet that you can to exceed the 150,000 mile rate... Here's how.

Now this is going out on a limb as I know more than the average bear about software and neural nets, but just enough to be dangerous about vehicle autonomy.

At the end of the day, it's all software, some procedural code, some NN, and all of it fed a stream of input data that controls servos. Devs already write tests for all of the procedural stuff daily. That's easy... But, I've recently read that the engineers at Wamo are now creating virtual environments used to train their NN systems so they can time-compress the training process...

Here is the limb... I bet that they have also devised their own internal test jigs for these virtual environments too. My prediction is that this training tool evolves into a test platform, one where real world scenario data, say, 150,001 miles worth, or billions in Tesla's case, would be captured and fed into these simulation jigs, and then the various manufacture computers and software could consume this artificial environment and be scored on how well they do.

Just a theory...

 

[Daniel in SD]

I partially agree. You are right, you can not contrive a test for every situation. But, I'd bet that you can to exceed the 150,000 mile rate... Here's how.

Now this is going out on a limb as I know more than the average bear about software and neural nets, but just enough to be dangerous about vehicle autonomy.

At the end of the day, it's all software, some procedural code, some NN, and all of it fed a stream of input data that controls servos. Devs already write tests for all of the procedural stuff daily. That's easy... But, I've recently read that the engineers at Wamo are now creating virtual environments used to train their NN systems so they can time-compress the training process...

Here is the limb... I bet that they have also devised their own internal test jigs for these virtual environments too. My prediction is that this training tool evolves into a test platform, one where real world scenario data, say, 150,001 miles worth, or billions in Tesla's case, would be captured and fed into these simulation jigs, and then the various manufacture computers and software could consume this artificial environment and be scored on how well they do.

Just a theory...

Seems like you should start your own autonomous vehicle company. Haha.
I've read there's already been $80 billion of VC money invested so far so it's probably not that hard to get funded!
At the autonomy day presentation Elon Musk didn't have great things to say about simulation which is something that other companies do a lot of.

 

[Bernesto]

Seems like you should start your own autonomous vehicle company. Haha.

I need more sleep than Elon seems too. Besides sleeping at the factory doesn't seem to glamorous. I'll just stick with armchair-quarterbacking lol

I can see both sides of simulation. On the one hand is theory, it should work, and work well. On the other hand, when does theory ever align with reality...

 

[Daniel in SD]

I need more sleep than Elon seems too. Besides sleeping at the factory doesn't seem to glamorous. I'll just stick with armchair-quarterbacking lol

I can see both sides of simulation. On the one hand is theory, it should work, and work well. On the other hand, when does theory ever align with reality...

Back in 2017 Waymo was simulating 8 million miles a day.
Waymo Built a Secret World for Self-Driving Cars

 

[Bernesto]

Back in 2017 Waymo was simulating 8 million miles a day.

Very cool, and valuable work. Good read. Thanks.