Tesla autonomy in 2020: two competing theories

[strangecosmos2]

I've noticed scrutinizers of Tesla's autonomy software have two theories about how Tesla's development process will unfold in 2020.

Theory 1: What You See Is What You Get (WYSIWYG)

This is the theory that the latest Autopilot/Full Self-Driving Capability update is already pretty close to what Tesla has in development. There is only a thin line between the software in production and the cutting edge. Future software updates will just bring slow, gradual, incremental updates on top of that.

Theory 2: Sudden Improvement

This is the theory that the dev software is far ahead of the production software. On this theory, Andrej Karpathy's team has developed new neural networks that are up to 10x larger than what's in customers' cars today and that perhaps have a new network architecture as well. New training datasets or techniques may be usable or more exploitable with the new nets. The result will be a sudden improvement when these new nets are deployed.

What do y'all think?

 

[powertoold]

During the latest YT talk, Karpathy talks about the "almost 100" tasks that the FSD NN has to prioritize. I don't think current AP is juggling 100 tasks, so Tesla is keeping the FSD stack in house and releasing bits and pieces. That's why I voted for sudden improvement once they deploy the whole stack.

There's no reason for them to deploy the whole FSD stack until it's fairly reliable, because most of the validation can be done in shadow mode.

 

[arroyobass]

I think the reliability of the system to an end user will be pretty similar. It's the really weird corner cases that still need to be ironed out. I'm pretty sure the software that is running in dev builds is very far ahead of what we have now, but it's not functioning at a high enough success rate to roll into the field. It's those extra 9's that the system really has to learn at this point.

They will push individual new features as they mature, so it won't be one big dump where one day we have full level 4 self driving.

 

[strangecosmos2]

They will push individual new features as they mature, so it won't be one big dump where one day we have full level 4 self driving.

I think everyone agrees with this part. Elon has said the first phase is “feature complete” full self-driving, which is a Level 2 system that can go anywhere.

 

[SandiaGrunt]

There's no reason for them to deploy the whole FSD stack until it's fairly reliable, because most of the validation can be done in shadow mode.

Tesla’s strategy is absolutely to wait until their FSD development branch is totally reliable, essentially incapable of any kind of failure, before doing an OTA to wake up the fleet of robotaxis. Knife-switch releases are clearly the most prudent and safe way to release software updates. Just look at their success with their Smart Summon release! Mothers and children everywhere feel safer when Tesla owners use Smart Summon instead of waking fifty feet to their cars.

 

[DanCar]

It won't be sudden improvement. They are accelerating in improvements, but they have hired a bigger team, but improvements will be linear.

 

[SandiaGrunt]

I'm pretty sure the software that is running in dev builds is very far ahead of what we have now, but it's not functioning at a high enough success rate to roll into the field. It's those extra 9's that the system really has to learn at this point.

Yes! Tesla is the only software company that has a development version that is more capable than their production version. They pioneered this form of software development, and it is just a matter of time before it all pays off.

Those extra 9’s of reliability are each just a single order of magnitude improvement in the failure rate, so it makes sense that whatever worked in the past will continue to work indefinitely in the future.

If Tesla needs a couple more 9’s of reliability, Trent says all they have to do is deploy a couple more factors of 1000x more cars. By the time they’ve deployed a few hundred billion cars, it’ll be checkmate, Waymo!

 

[heltok]

Team sudden improvement. Higher resolution, more cameras sharing features, multiple frames, lstm network and hydranet using resources more efficiently. Autonomy day demo seemed capable, since then they have had HW3 cars in shadow mode collecting tons of data and project vacation and Dojo patching holes for almost a year. I expect it will be ELO 1500 to 2000 in bullet kind of improvement.

 

[diplomat33]

I think the truth is probably somewhere in the middle. It's the nature of research and development that the research team have stuff more advanced than what the public has. So it stands to reason that the Tesla development team have more advanced neural nets and more more advanced software than what we have now in our cars, although probably not light years more advanced. And the incremental release of features would seem to suggest that Tesla releases pieces as they validate each piece.

 

[eli_]

A big engineering project like this is going to have probably hundreds or thousands of deliverables that individuals or small teams are working on all at once, and they're not going to all drop at the same time. It's not going to be one big monolithic thing, folks will want to ship their code even turned off, and just verify things are working. They could do that by packaging it as a new customer-facing feature like "Smart Summon", or just have it run quietly in the background and collect data for a few months. There's probably a huge development effort just for internal tooling and engineering that we'll never have visibility into, tools for ML training, tools for regression tests (i.e. Simulator), tools for debugging and visualizing customer telemetry. What they work on internally could look totally different from what customers see in terms of visible features.

I will now use my imagination and dream up what their work items might look like just to put a finer point on the vast scope of this effort...

• Vision now handles large vehicles crossing multiple cameras
• Model trained to classify parking stalls for Smart Summon/Autopark, etc.
• Smart Summon planner version 2
• Ship v1 of predictive automatic emergency braking
  
• Telemetry system is extended to support more advanced triggers/campaigns
  
• Lane localization version 5.0
• Better health monitoring for obstructed cameras
  
• Tow trucks added to driving Simulator
• Simulator now supports Japanese traffic signs
  
• Better tools for data set labeling
• Initial engineering tools for HW4
  
• ADAS visualization now renders stop lines and traffic lights
• New policy for lane positioning
• Improvements to NoA planner for some obscure type of highway fork in Australia
  
• New controller for stopping at crosswalks
• ... And 1000 other things

No way all these things drop at once. Eventually they could hit some internal milestone like "City Driving 1.0", but it might depend on some hundreds of other features getting done and it's going to be this long, multi-year, incremental journey getting there. A lot might be hidden from us, but we'll probably see some of the steps visibly happen in future builds, given how often they ship.

 

[strangecosmos2]

Thanks for this great comment, @eli_. Here, have a gold:

 

[diplomat33]

A big engineering project like this is going to have probably hundreds or thousands of deliverables that individuals or small teams are working on all at once, and they're not going to all drop at the same time. It's not going to be one big monolithic thing, folks will want to ship their code even turned off, and just verify things are working. They could do that by packaging it as a new customer-facing feature like "Smart Summon", or just have it run quietly in the background and collect data for a few months. There's probably a huge development effort just for internal tooling and engineering that we'll never have visibility into, tools for ML training, tools for regression tests (i.e. Simulator), tools for debugging and visualizing customer telemetry. What they work on internally could look totally different from what customers see in terms of visible features.

I will now use my imagination and dream up what their work items might look like just to put a finer point on the vast scope of this effort...

• Vision now handles large vehicles crossing multiple cameras
• Model trained to classify parking stalls for Smart Summon/Autopark, etc.
• Smart Summon planner version 2
• Ship v1 of predictive automatic emergency braking
  
• Telemetry system is extended to support more advanced triggers/campaigns
  
• Lane localization version 5.0
• Better health monitoring for obstructed cameras
  
• Tow trucks added to driving Simulator
• Simulator now supports Japanese traffic signs
  
• Better tools for data set labeling
• Initial engineering tools for HW4
  
• ADAS visualization now renders stop lines and traffic lights
• New policy for lane positioning
• Improvements to NoA planner for some obscure type of highway fork in Australia
  
• New controller for stopping at crosswalks
• ... And 1000 other things

No way all these things drop at once. Eventually they could hit some internal milestone like "City Driving 1.0", but it might depend on some hundreds of other features getting done and it's going to be this long, multi-year, incremental journey getting there. A lot might be hidden from us, but we'll probably see some of the steps visibly happen in future builds, given how often they ship.

Great comment!!

 

[MasterchiefTS]

A big engineering project like this is going to have probably hundreds or thousands of deliverables that individuals or small teams are working on all at once, and they're not going to all drop at the same time. It's not going to be one big monolithic thing, folks will want to ship their code even turned off, and just verify things are working. They could do that by packaging it as a new customer-facing feature like "Smart Summon", or just have it run quietly in the background and collect data for a few months. There's probably a huge development effort just for internal tooling and engineering that we'll never have visibility into, tools for ML training, tools for regression tests (i.e. Simulator), tools for debugging and visualizing customer telemetry. What they work on internally could look totally different from what customers see in terms of visible features.

I will now use my imagination and dream up what their work items might look like just to put a finer point on the vast scope of this effort...

• Vision now handles large vehicles crossing multiple cameras
• Model trained to classify parking stalls for Smart Summon/Autopark, etc.
• Smart Summon planner version 2
• Ship v1 of predictive automatic emergency braking
  
• Telemetry system is extended to support more advanced triggers/campaigns
  
• Lane localization version 5.0
• Better health monitoring for obstructed cameras
  
• Tow trucks added to driving Simulator
• Simulator now supports Japanese traffic signs
  
• Better tools for data set labeling
• Initial engineering tools for HW4
  
• ADAS visualization now renders stop lines and traffic lights
• New policy for lane positioning
• Improvements to NoA planner for some obscure type of highway fork in Australia
  
• New controller for stopping at crosswalks
• ... And 1000 other things

No way all these things drop at once. Eventually they could hit some internal milestone like "City Driving 1.0", but it might depend on some hundreds of other features getting done and it's going to be this long, multi-year, incremental journey getting there. A lot might be hidden from us, but we'll probably see some of the steps visibly happen in future builds, given how often they ship.

With "Initial engineering tools for HW4" you mean autopilot 3?

 

[eli_]

With "Initial engineering tools for HW4" you mean autopilot 3?

No they mentioned there's some future version at the Autonomy Day presentation earlier this year, so I'm just taking that to mean there's a HW4 coming, but it's a long way off.

 

[MasterchiefTS]

No they mentioned there's some future version at the Autonomy Day presentation earlier this year, so I'm just taking that to mean there's a HW4 coming, but it's a long way off.

Yep. I can't wait the work on Hardware 3 The next gen is very good

 

[mswlogo]

Poll kind of skimpy.

gradual steady progress with some setbacks and some bursts of significant features.

Sort of what has happened in the past few months.

I expect it’s gonna be a great year now that they are starting to use HW3 and cars are getting upgraded.

 

[MasterchiefTS]

It would be cool but I have some worries for release of FSD in 2020 :/ Hardware 3 should be the state of art for FSD, We will se how the developers developing on this hardware