Profound progress towards FSD

[DDotJ]

That's the same source that posted the other leaks awhile back about V11.

Yeah, I'm guessing the new FSD features that are 1-2 months out are part of v11.

I wonder about the performance improvements to AP with this new rewrite. I wonder if we'll see improvements to driving policy and features like Smart Summon and NoA.

It also looks like 2020.36 also shows the 3D renders of the speed limit signs in the visualizations.

 

[diplomat33]

I wonder about the performance improvements to AP with this new rewrite. I wonder if we'll see improvements to driving policy and features like Smart Summon and NoA.

I am hoping to see some big performance improvements. But I am not sure about driving policy since the rewrite focuses on perception. But hopefully, Tesla will add improved driving policy on top of the rewrite.

 

[DDotJ]

I am hoping to see some big performance improvements. But I am not sure about driving policy since the rewrite focuses on perception. But hopefully, Tesla will add improved driving policy on top of the rewrite.

I was under the impression that the AP Rewrite was more than just perception. I remember Elon saying that they were combining planning and perception instead of having them as separate components.

There's quite a significant foundational rewrite in the Tesla Autopilot system that's almost complete. … Instead of having planning, perception, image recognition all be separate, they're being combined. … Effectively, the neural net is absorbing more and more of the problem beyond simply if you see an image: is this a car or not? It's kind of: what do you do from that?

3D labeling is the next big thing, where the car can go through a scene with eight cameras and kind of paint a path and then you can label the path in 3D. This is probably a two or three order of magnitude improvement in labeling efficiency...

Source

 

[diplomat33]

I was under the impression that the AP Rewrite was more than just perception. I remember Elon saying that they were combining planning and perception instead of having them as separate components.

You are correct. The rewrite will combine perception with some planning.

 

[DDotJ]

You are correct. The rewrite will combine perception with some planning.

Ah, I must be getting some terminology mixed up. I thought driving policy and planning were interchangeable terms.

Would NoA lane change decisions on a highway and moving around a delivery truck on a two way road be part of planning? Or is planning strictly just drawing a path / trajectory within an environment?

 

[diplomat33]

Ah, I must be getting some terminology mixed up. I thought driving policy and planning were interchangeable terms.

Would NoA lane change decisions on a highway and moving around a delivery truck on a two way road be part of planning? Or is planning strictly just drawing a path of travel within an environment?

Driving Policy is all the rules that dictate how the autonomous car will drive. Planning is the car figuring what to do in a given situation based on perception. So planning is drawing a safe path but also deciding when to brake or accelerate.

For example, the NOA auto lane change settings (no, mild, normal, mad max) is Driving Policy. That driving policy then influences Planning. Driving policy decides when to do an auto lane change. Planning decides how to do the auto lane change. So planning is figuring out the safe path for the auto lane change as well as when to brake or accelerate.

 

[Grifon]

I think something to look for is evidence that the car is seeing in 4D. We need to keep an eye out for when it appears the car can see with 1 360 degree view vs. 8 individual views from each camera. The current version of AP for me (32.2) still sees some objects at times as 2 distinct cones, trash cans, etc. and will show that on the display. I believe that’s why we still see dancing cars and trucks. The new 4D version is supposed to do a better job of stitching those 8 video feeds together in real time to better detect objects as single or multiple entities as appropriate.

 

[DDotJ]

Driving Policy is all the rules that dictate how the autonomous car will drive. Planning is the car figuring what to do in a given situation based on perception. So planning is drawing a safe path but also deciding when to brake or accelerate.

For example, the NOA auto lane change settings (no, mild, normal, mad max) is Driving Policy. That driving policy then influences Planning. Driving policy decides when to do an auto lane change. Planning decides how to do the auto lane change. So planning is figuring out the safe path for the auto lane change as well as when to brake or accelerate.

Ah that makes perfect sense, thanks for the clarification. I guess my chief complaints with NoA are with driving policy, I hope Tesla improves on that front.

 

[diplomat33]

I think something to look for is evidence that the car is seeing in 4D. We need to keep an eye out for when it appears the car can see with 1 360 degree view vs. 8 individual views from each camera. The current version of AP for me (32.2) still sees some objects at times as 2 distinct cones, trash cans, etc. and will show that on the display. I believe that’s why we still see dancing cars and trucks. The new 4D version is supposed to do a better job of stitching those 8 video feeds together in real time to better detect objects as single or multiple entities as appropriate.

Yes, but keep in mind that the rewrite is in shadow mode in 2020.36 so it is not actually "live". It is not controlling the car yet. So I doubt we will see any difference. We have to wait for the rewrite to be turned on before we see any changes. If I had to guess, I suspect the rewrite might go live in 2020.40.

 

[Knightshade]

Proof that FSD is a solved problem:

https://twitter.com/elonmusk/status/1298350042894327808

If Elon can terraform Mars, FSD will be a doddle.

Yeah he really can't though.

Any created atmosphere will just leak back into space like the original did without the planet having a magnetic field to protect it.

See also:
These trippy Martian visuals hold clues to why the planet lost its atmosphere

Elon has hand-waived this away in the past saying he'll just make one which is... certainly a thing you'd want to do...but not a thing we can currently do, and you're likely talking generations before we could.


So- much like FSD- he insists he can make a theoretical thing nobody's ever done before happen before he's really figured out most of the details... and much like FSD he's probably going to find most of his original time estimates wildly optimistic.

 

[powertoold]

he can make a theoretical thing nobody's ever done before happen before he's really figured out most of the details...

This would be more convincing if Elon didn't head multiple highly-detail-oriented companies (possibly the *most* detail-oriented companies known to mankind) that have produced demonstrable results.

 

[john5520]

Yes, but keep in mind that the rewrite is in shadow mode in 2020.36 so it is not actually "live". It is not controlling the car yet. So I doubt we will see any difference. We have to wait for the rewrite to be turned on before we see any changes. If I had to guess, I suspect the rewrite might go live in 2020.40.

Yeah, I was getting all excited before I read about shadow mode. I least I should be able to see what speed it's detecting.

 

[cwerdna]

View attachment 576452 A year or so....guess that cross country FSD won’t be happening anytime soon :/

Thank you!

"Year or so" tweet at https://twitter.com/elonmusk/status/1294375897319890944?s=20.
Copy at https://archive.is/Nlm7A.

"4D" tweet at https://twitter.com/elonmusk/status/1294375249761275904?s=20
Copy at https://twitter.com/elonmusk/status/1294375249761275904?s=20.

 

[NHK X]

Yes, but keep in mind that the rewrite is in shadow mode in 2020.36 so it is not actually "live". It is not controlling the car yet. So I doubt we will see any difference. We have to wait for the rewrite to be turned on before we see any changes. If I had to guess, I suspect the rewrite might go live in 2020.40.

Is shadow mode real? I believed it is but I thought green said it doesn’t exist.

 

[scottf200]

Various reddit post about this being the first drop of the 're-write' code. Also, it will be great to get this out there and have them get 1000s and 1000s of test/verifications (ie. compare what the computer suggest to what the human actually does (chime when you were in red arrowed lane)).

2020.36 Release Notes includes Green Traffic Light Chime and Speed Limit Sign Recognition!! : teslamotors

 

[diplomat33]

Is shadow mode real? I believed it is but I thought green said it doesn’t exist.

Shadow Mode is real but I think there has been some misconceptions about what it does. But Tesla has mentioned it specifically by name, most notably during Autonomy Day:

"Our next generation automatic emergency braking which will not just stop for people in your path but also stop for people that are going to be in your path and that's running in shadow mode right now and we'll go out to the fleet this quarter. I'll talk about shadow mode in a second."

Shadow Mode was described this way during Autonomy Day:

"When we initially have some algorithms that we want to try out, we can put them on the fleet and we can see what they would have done in a real world scenario such as this car that is overtaking us very quickly. And this is taken from our actual simulation environment, showing different paths that we considered taking and how those overlay with real world behavior of a user."

 

[Knightshade]

This would be more convincing if Elon didn't head multiple highly-detail-oriented companies (possibly the *most* detail-oriented companies known to mankind) that have produced demonstrable results.

I mean- it's not like he's done nothing.

But he's also consistently made promises years beyond the original promised deliveries.

In particular anytime he makes a claim or promise about something he has never actually done he's always extremely late on delivering- and in many cases still hasn't delviered.

We're what 4 years out from the promise of a cross country FSD drive by end of year?

Elon himself admits he has no idea how long stuff he's not actually done will take...he said it regarding the Model 3 ramp and it going poorly and slowly in the early days-

People should not ascribe to malice that which can easily be explained by stupidity." (LAUGHTER) So-- so it's, like, just because I'm, like, dumb at-- at predicting dates does not mean I am untruthful. I don't know-- I-- we've-- I never made a mass-produced car. How am I supposed to know with precision when it's gonna get done

Now sub "FSD" for "made a mass produced car"

Or sub "create a magnetic field on another planet" or "create a breathable atmosphere on another planet"


And unlike mass producing a car- which is something other humans have actually done all the other things he's promising nobody has ever done, and require a lot more tech that does not yet even exist (the mars stuff in particular)


I believe Elon believes he can do the mars stuff he says he can.

I do not believe it's, even remotely, anything like "solved" as previously claimed here.

Neither even is FSD at this point- though it's certainly closer than "create a magnetosphere from scratch on Mars"

 

[powertoold]

In particular anytime he makes a claim or promise about something he has never actually done he's always extremely late on delivering- and in many cases still hasn't delviered.

There aren't many things on this list anymore.

Also, I'm in your camp. I think FSD is basically impossible given our current NN technology. I don't believe it either, and I work with software and follow machine learning.

But I think Elon is well-aware of the details required for FSD. He thrives in detail and precision in all his endeavors, so it's difficult to believe he doesn't understand the FSD problem.

 

[diplomat33]

I mean- it's not like he's done nothing.

But he's also consistently made promises years beyond the original promised deliveries.

In particular anytime he makes a claim or promise about something he has never actually done he's always extremely late on delivering- and in many cases still hasn't delviered.

We're what 4 years out from the promise of a cross country FSD drive by end of year?

Elon himself admits he has no idea how long stuff he's not actually done will take...he said it regarding the Model 3 ramp and it going poorly and slowly in the early days-




Now sub "FSD" for "made a mass produced car"

Or sub "create a magnetic field on another planet" or "create a breathable atmosphere on another planet"


And unlike mass producing a car- which is something other humans have actually done all the other things he's promising nobody has ever done, and require a lot more tech that does not yet even exist (the mars stuff in particular)


I believe Elon believes he can do the mars stuff he says he can.

I do not believe it's, even remotely, anything like "solved" as previously claimed here.

Neither even is FSD at this point- though it's certainly closer than "create a magnetosphere from scratch on Mars"

Elon recently tweeted this quote from Ernest Rutherford:

I think this quote explains why Elon keeps saying things are "solved" when they are not. He clearly subscribes to this Rutherfordian view that things become trivial once you understand the underlying physics. So since he feels he understands the underlying physics of FSD, it is "solved". Since he feels he understands the underlying physics of terraforming Mars, it is "solved". Elon seems to focus more on understanding problems and finding solutions. Once he feels like he knows the solution, it is "solved" in his mind because then it is just a matter of "doing the work" to implement the solution.

But there is a big gap between theory and a practical application. Even after you understand the theory, engineering a practical solution requires a lot of work and solving a lot of new problems that you did not think about when you were just working on the theory part. We've seen this play out with FSD. Elon feels he understands the physics behind FSD like doing ML to solve vision. But actually doing the work has taken longer than he thinks, requiring multiple rewrites and solving new problems.

 

[Daniel in SD]

Elon recently tweeted this quote from Ernest Rutherford:

I think this quote explains why Elon keeps saying things are "solved" when they are not. He clearly subscribes to this Rutherfordian view that things become trivial once you understand the underlying physics. So since he feels he understands the underlying physics of FSD, it is "solved". Since he feels he understands the underlying physics of terraforming Mars, it is "solved".

But there is a big gap between theory and a practical application. Even after you understand the theory, engineering a practical solution requires a lot of work and solving a lot of new problems that you did not think about when you were just working on the theory part. We've seen this play out with FSD. Elon feels he understands the physics behind FSD like doing ML to solve vision. But actually doing the work has taken longer than he thinks, requiring multiple rewrites and solving new problems.

I understand the underlying physics of simply moving Mars closer to the sun. Solved?