Autonomous Car Progress

[stopcrazypp]

Elon Musk says full self-driving Tesla tech 'very close'

A future software update could activate full "level-five" autonomy in cars, the Tesla founder says.

www.bbc.com

"I remain confident that we will have the basic functionality for level five autonomy complete this year.
"There are no fundamental challenges remaining.

A year ago today.

What "basic functionality" means above is the critical point.

Elon touched on that on Autonomy day in 2019.
"There’s 3 steps to self-driving: there’s feature complete, then there’s feature complete to the degree that … where we think that the person in the car does not need to pay attention, then there’s at a reliability level where we also convince regulators that that is true"
Elon Musk says Tesla will have 1 million robo-taxis on the road next year, and some people think the claim is so unrealistic that he's being compared to PT Barnum

So in Elon's book "feature complete" is essentially release of "City Streets" AKA end to end L2 as some people call it. "basic functionality" likely refers to the same thing. It seems he expected to release that feature by end of 2020 and that obviously didn't happen. It's not clear if that feature would be ready to release even by the end of this year.

There's still a huge gap between end-to-end L2 and L5 and it seems Elon underestimates the challenge there (it's possible to have a great end-to-end L2 system and never reach L5).

 

[daniel]

Might depend on the state. Looking at Nevada's DMV form for public use autonomous vehicles, it looks to be "self-certification." Just check off the box that says "Is capable of operating in compliance with all applicable motor vehicle laws and traffic laws of this State" along with a bunch of other bits on a 2 page form and e-mail it in.

Well, that's not good.

Hopefully, carmakers will not put autonomous cars in service until they can exceed the safety of human drivers. Some might be more responsible than others. I'm glad I don't live in Nevada or Florida. (For many reasons, but this is one more.)

 

[Microterf]

Does anyone know if where FSD 9 is in terms of the stack rewrite? Is that complete now and it should just be training iterations from here on out, and we'll know if it isn't capable of getting there relatively quickly, or is there still a bunch that needs transitioned to "Software 2.0". I felt like in Oct when they announced FSD beta, that it that was already done, then we learned they had to rewrite the rewrite. Just trying to get the correct expectation.

Maybe @verygreen or @diplomat33

 

[verygreen]

Does anyone know if where FSD 9 is in terms of the stack rewrite? Is that complete now and it should just be training iterations from here on out, and we'll know if it isn't capable of getting there relatively quickly, or is there still a bunch that needs transitioned to "Software 2.0". I felt like in Oct when they announced FSD beta, that it that was already done, then we learned they had to rewrite the rewrite. Just trying to get the correct expectation.

Maybe @verygreen or @diplomat33

you know that FSD beta 9 was not released yet I hope?

 

[Daniel in SD]

Does anyone know if where FSD 9 is in terms of the stack rewrite? Is that complete now and it should just be training iterations from here on out, and we'll know if it isn't capable of getting there relatively quickly, or is there still a bunch that needs transitioned to "Software 2.0". I felt like in Oct when they announced FSD beta, that it that was already done, then we learned they had to rewrite the rewrite. Just trying to get the correct expectation.

Maybe @verygreen or @diplomat33

I interpret "there" to mean twice as safe as human driver average in the US. There is no way for us to determine if it's "there" since that would require millions of miles of data. I assume that it will be clear on Saturday that it's not "there" but how would we know if the new architecture is capable of getting "there"?

 

[Microterf]

I interpret "there" to mean twice as safe as human driver average in the US. There is no way for us to determine if it's "there" since that would require millions of miles of data. I assume that it will be clear on Saturday that it's not "there" but how would we know if the new architecture is capable of getting "there"?

Yes. Sorry, was writing that in a hurry. Yes, by getting there, I'm talking about getting to superhuman level driving.

I know it's not out yet, I'm asking if anyone had any idea if this was the final architecture, and at this point it'd be a matter of training, or if there are further architectural changes they plan on making. (Moving more of the stack to software 2.0)

 

[JHCCAZ]

This comment is a little out on our limb because I have no access to the inside info or software as some others do. But I interpret the sequence of events this way:

The newest software 2.0 dev track was being applied to City Streets aka FSDbeta in 2020, while older highway-centric AP, TACC, NoA etc. were receiving relatively lesser maintenance/dev effort. Planned for later work. But then, maybe early 1Q, the realization of needing to reallocate engineering resources to cope with radar-removal (for whatever reasons as we've all debated) forced a retrofit of the older highway and standard AP. Hence the obvious rewrite and requalification scrambling, abandonment of the promised FSDbeta v8.3 release and the big pause in FSDbeta v9 release.

Now, again I couldn't say for sure that this AP rewrite was done under the new Karpathy-led Software 2.0 paradigm, or more a traditional code rewrite of AP to remove radar data input. But I suspect the former based on Karpathy's presentation. If so that's a good thing as it addresses the somewhat neglected AP features that everyone actually has, and brings them sooner into better parity with the purported overall Vision FSD.

Maybe @verygreen or others can support or contradict.

 

[uujjj2]

It's as much a compute problem as a "software" problem or a "data" problem. I think this is something Karpathy mentioned in one of his presentations. Tesla has a NN (really something with multiple connected NNs plus other code) that can do full self driving pretty well, but it can only run in their datacenter and not the low-power computer in the cars. They have a distilled version of the NN that runs in the cars that's not as good.

And I think it will be 2 or 3 more versions of the in-car computer before we have robust full self driving able to run in-car, which is why the Tesla chip team are still hard at work and not collecting retirement.

 

[uujjj2]

Look at how regular highway autopilot has evolved over the last almost 6 years. The original hands-off v7 autopilot is better than the latest version, at least as far as the driver perceives it. It's smoother, more confident, let's you move the car around within a lane without disengaging. But it's also an order of magnitude less safe than the latest autopilot.

Behind the scenes, autopilot may be making huge leaps, but as far as the customer sees (aside from crashes) it's stagnated if not gotten slightly worse.

So that's interesting, and I wonder if full self driving (which IMHO should be called partial self driving) is going to evolve the same way. It will be autopilot on city streets, hands on driver needs to keep paying attention. But it's still really freaking impressive, and will blow customers' minds when it comes out. But to customers' eyes it'll seemingly stagnate for years, the only improvements being behind-the-scenes safety improvements, until one day 3, 5, 10 years from now where the safety finally reaches a threshold and Tesla+DMV finally allows it to be hands-off eyes-off.

 

[Daniel in SD]

until one day 3, 5, 10 years from now where the safety finally reaches a threshold and Tesla+DMV finally allows it to be hands-off eyes-off.

I don't believe this will work. People will stop being vigilant long before it reaches human level safety. If the car goes a whole year without making a mistake (15k miles) are you really going to be paying enough attention to stop it from running over a pedestrian in year 2?
Google had the same plan until they decided it wouldn't work.

 

[rxlawdude]

Look at how regular highway autopilot has evolved over the last almost 6 years. The original hands-off v7 autopilot is better than the latest version, at least as far as the driver perceives it. It's smoother, more confident, let's you move the car around within a lane without disengaging.

I assume you have objective proof of this whopper!

 

[mikes_fsd]

A 2-day old comment by Karpathy that I have not seen posted here -- he has talked about that a diverse and well labeled data set is of HUGE importance, so the fact that managing the flow (and quality) of the work is only now getting into a manageable workflow is telling just how massive that task is.

Also, further down in that thread he answers about using simulation:

src:

https://twitter.com/x/status/1413258221096103938

 

[diplomat33]

Also, further down in that thread he answers about using simulation:
View attachment 683321
src:

https://twitter.com/x/status/1413258221096103938

The "joe" guy does not understand how Waymo uses sims. Sims do not replace getting real world data. And Waymo does not use sims for labeling. Waymo is literally building entire realistic virtual cities to evaluate and test the software.

"At Waymo, we use simulation to advance the Waymo Driver's abilities in several ways. We can amplify our number of real-world miles to accelerate our Driver’s learning, prepare for rare events, train new models, validate new software, and even evaluate how the Waymo Driver would have performed in actual fatal crashes that have already occurred."

Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving

At Waymo, we use simulation to advance the Waymo Driver's abilities in several ways. We can amplify our number of real-world miles to accelerate our Driver’s learning, prepare for rare events, train new models, validate new software, and even evaluate how the Waymo Driver would have performed in...

blog.waymo.com

 

[mikes_fsd]

The "joe" guy does not understand how Waymo uses sims.

You are focusing on the wrong thing.
That is why my screenshot left his stuff out.

What Karpathy said is what matters in this context, not what Joe-shmoe said.

 

[diplomat33]

You are focusing on the wrong thing.
That is why my screenshot left his stuff out.

What Karpathy said is what matters in this context, not what Joe-shmoe said.

And what Karpathy said does not really make sense either. He says Tesla only uses sims for labeling. What? Maybe Karpathy is thinking of something else. Waymo uses sims to evaluate the software, not labeling. Waymo is building entire virtual cities to test the software.

Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving

At Waymo, we use simulation to advance the Waymo Driver's abilities in several ways. We can amplify our number of real-world miles to accelerate our Driver’s learning, prepare for rare events, train new models, validate new software, and even evaluate how the Waymo Driver would have performed in...

blog.waymo.com

 

[mikes_fsd]

And what Karpathy said does not really make sense either. He says Tesla only uses sims for labeling. What?

Oh dear Lord, you cannot separate what he is answer to, vs what he is saying about pro's and con's.

In his CVPR 2021 presentation he said they used simulation for testing as well to get Tesla Vision into production.

 

[uujjj2]

I assume you have objective proof of this whopper!

I'm not claiming autopilot v7 on the 1.0 hardware is actually better than the latest autopilot. But a naive customer, in the driver's seat, will perceive the earlier version as better because it allows hands free driving, is just about as smooth, and seems more confident than today's autopilot, even though v7 autopilot is also a lot more likely to plow you into a truck.

 

[ZeApelido]

The "joe" guy does not understand how Waymo uses sims. Sims do not replace getting real world data. And Waymo does not use sims for labeling. Waymo is literally building entire realistic virtual cities to evaluate and test the software.

"At Waymo, we use simulation to advance the Waymo Driver's abilities in several ways. We can amplify our number of real-world miles to accelerate our Driver’s learning, prepare for rare events, train new models, validate new software, and even evaluate how the Waymo Driver would have performed in actual fatal crashes that have already occurred."

Simulation City: Introducing Waymo's most advanced simulation system yet for autonomous driving

At Waymo, we use simulation to advance the Waymo Driver's abilities in several ways. We can amplify our number of real-world miles to accelerate our Driver’s learning, prepare for rare events, train new models, validate new software, and even evaluate how the Waymo Driver would have performed in...

blog.waymo.com

The "joe" fully understands how Waymo uses sims. Which is why he understands the benefits and limitations of sims. He understands that more sims do not replace the needs of sufficient real world data - that is the point of disagreement with Waymo - who push the narrative that they are collecting enough real world data and mostly show off fancy simulations.

Data scientists understand collecting data from 100s of cars in a few cities won't likely be enough real world data for deployment say everywhere in the U.S. It's just not diverse enough.

 

[diplomat33]

The "joe" fully understands how Waymo uses sims. Which is why he understands the benefits and limitations of sims. He understands that more sims do not replace the needs of sufficient real world data - that is the point of disagreement with Waymo - who push the narrative that they are collecting enough real world data and mostly show off fancy simulations.

Data scientists understand collecting data from 100s of cars in a few cities won't likely be enough real world data for deployment say everywhere in the U.S. It's just not diverse enough.

No, "joe" does not understand how Waymo uses sims because he said that Waymo is claiming that sims can replace ALL real world data which is patently false. Waymo is not claiming that you don't need any real world data at all. Waymo is collecting real world data and relies on real world data. They uses sims in addition to real world data. Waymo is using sims to complement their real world data, not replace it. Waymo uses sims to test and validate full trips in different conditions. They are not using sims to replace all real world data as "joe" claims.

 

[ZeApelido]

No, "joe" does not understand how Waymo uses sims because he said that Waymo is claiming that sims can replace ALL real world data which is patently false. Waymo is not claiming that you don't need any real world data at all. Waymo is collecting real world data and relies on real world data. They uses sims in addition to real world data. Waymo is using sims to complement their real world data, not replace it. Waymo uses sims to test and validate full trips in different conditions. They are not using sims to replace all real world data as "joe" claims.

No one believes Waymo thinks they can do it all based on simulation.

People do believe Waymo understates the requirements of real world data acquisition. I believe Waymo's wording makes it sound like simulation can replace the lack of additional data they do not have. Additional, not all.