FSD rewrite will go out on Oct 20 to limited beta

[Knightshade]

My reasoning is that although the visualization detects the parked cars correctly, the car still plans a path that would crash into a parked vehicle, therefore the planner does not explicitly draw a path through free space using a "software 1.0" planner. One possible way this could happen is if the visualization was using a different unrelated network than the planner, so their decisions would not necessarily coincide. Since I know that Dojo and end-to-end learning is currently in progress, I think the current planner is doing something akin to end-to-end, but maybe with a shorter amount of video as input to this planner. For example, the neural network could take the last 1-2 seconds of video recorded as input into the planner network instead of the last 20 seconds. This would allow them to train the planner network without having Dojo, but the downside is that to make a good planner you likely need more than the past 1-2 seconds.

FWIW according to green even the FSD beta still uses conventional C++ code for everything other than perception....planning in the sense of "make a best guess about how the road I can't see over that hill will curve" is perception too.... but actual driving policy that decides what the car actually does is still done in conventional code.

There's 58 NNs total operating right now.... and most of it is the same code everyone is running... the only things that were "added" for the FSD beta are 4 more NNs and a whole bunch of conventional c++ code in a module named city_streets

(which is why the idea this was a "total rewrite" is....not terribly accurate... many of the NNs in the FSD beta are the most-current versions of ones that have been in use since 2018, and many more since 2019)

 

[emmz0r]

FWIW according to green even the FSD beta still uses conventional C++ code for everything other than perception....planning in the sense of "make a best guess about how the road I can't see over that hill will curve" is perception too.... but actual driving policy that decides what the car actually does is still done in conventional code.

There's 58 NNs total operating right now.... and most of it is the same code everyone is running... the only things that were "added" for the FSD beta are 4 more NNs and a whole bunch of conventional c++ code in a module named city_streets

(which is why the idea this was a "total rewrite" is....not terribly accurate... many of the NNs in the FSD beta are the most-current versions of ones that have been in use since 2018, and many more since 2019)

It's gonna take a long time till policies are NN and frankly it doesn't make sense yet. For simple feedback loop stuff, sure, but for complex decisions, no. It would also be incredibly hard / impossible to actually debug.

It's not a total rewrite, but the way of processing it might be. The stitching and processing based on what that produces in a true 3D projection is the rewrite.

 

[turnem]

After watching a lot of these videos, here are my thoughts on current state of FSD:

1. I don’t anticipate a public release within a few months. I hope not at least. I’d like to see very few interventions amongst the beta group before it goes to a wider release.

2. I see no evidence yet of a sensor problem. I think the majority of the problems are vehicle path planning problems, which is very complicated. I am confident that we’ll see steady improvement over time, which will be really fun to watch.

Are we close to true FSD? No way, not yet.

Am I excited about where this is headed? Absolutely. Let’s remind ourselves that FSD is probably the most complicated engineering and CS problem ever conceived. Not expecting it to be a fast process.

I agree with all of this except that I DO hope we see a broader public release of FSD soon. What I'm seeing in the videos is still better than what I have in my software version. I use autopilot every time I get in the car (including on city streets) and I have to take over at least as frequently as these beta testers are having to take over... maybe more.

I'm not super educated on how NN works but it seems to me that the more feedback it gets the quicker it can improve. So I'd like to see them roll this out publicly in order to feed the machine.

I wouldn't say that if I felt that the current beta was less safe than what we have now but I think it's at least equally as safe if not slightly improved.

 

[Todd Burch]

I agree with all of this except that I DO hope we see a broader public release of FSD soon. What I'm seeing in the videos is still better than what I have in my software version. I use autopilot every time I get in the car (including on city streets) and I have to take over at least as frequently as these beta testers are having to take over... maybe more.

I'm not super educated on how NN works but it seems to me that the more feedback it gets the quicker it can improve. So I'd like to see them roll this out publicly in order to feed the machine.

I wouldn't say that if I felt that the current beta was less safe than what we have now but I think it's at least equally as safe if not slightly improved.

Well that's true. I just want the process to be safe, both for Tesla's sake and to protect the future of the technology itself.

I have to admit that given this is city driving, speeds are slower so there is less risk than highway driving. (Although there is a higher risk for dangerous pedestrian accidents). It's just a reminder that ALL drivers using the system will need to continue to remain vigilant for years.

 

[Knightshade]

I agree with all of this except that I DO hope we see a broader public release of FSD soon. What I'm seeing in the videos is still better than what I have in my software version. I use autopilot every time I get in the car (including on city streets) and I have to take over at least as frequently as these beta testers are having to take over... maybe more.

Probably because the current system explicitly tells you in the owners manual it's not intended to be used on city streets.

That's user error.

I
I'm not super educated on how NN works but it seems to me that the more feedback it gets the quicker it can improve. So I'd like to see them roll this out publicly in order to feed the machine.

That's not how that works at all though.

The data used to train the fleet is passively collectable from all vehicles, regardless of if they have a particular FSD version or even if they have the system turned on.

Good explanation here-
https://twitter.com/greentheonly/status/1096322810694287361?lang=en


The only extra stuff happening with the FSD beta is the manual "send this situation including video to Tesla" button the early access folks get.... they can't do that fleetwide as it requires a human manually reviewing and acting on that data which Tesla doesn't have the manpower to do for the whole fleet (hence why that button has never appeared for normal users)

 

[turnem]

Probably because the current system explicitly tells you in the owners manual it's not intended to be used on city streets.

That's user error.




That's not how that works at all though.

The data used to train the fleet is passively collectable from all vehicles, regardless of if they have a particular FSD version or even if they have the system turned on.

Good explanation here-
https://twitter.com/greentheonly/status/1096322810694287361?lang=en


The only extra stuff happening with the FSD beta is the manual "send this situation including video to Tesla" button the early access folks get.... they can't do that fleetwide as it requires a human manually reviewing and acting on that data which Tesla doesn't have the manpower to do for the whole fleet (hence why that button has never appeared for normal users)

Glad to see you are still fully dedicated to your same old tired rhetoric!

But you're kind-of making my point. We ALL know that a lot of folks use Autopilot on city streets. Heck, the current software both detects and reacts to stop signs and stop lights. Those certainly aren't found on any freeways that I drive on.

So regardless of whether you think it's use error for anyone to use autopilot on city streets (and I clearly think you're wrong) I think it's fairly evident that FSD is much safer on city streets than the current software. So... I'm ready to see it.

Thanks for the information on how the fleet data is used.

 

[Knightshade]

Glad to see you are still fully dedicated to your same old tired rhetoric!

But you're kind-of making my point. We ALL know that a lot of folks use Autopilot on city streets.

Yeah, a lot of folks drive drunk too.

Or don't bother checking their blind spots making lane changes.

Or don't check their tire pressure even when the light comes on.

All driver error though. The fact we "know" folks do it doesn't really change that.

G

So regardless of whether you think it's use error for anyone to use autopilot on city streets (and I clearly think you're wrong)

This has nothing to do with what either of us "thinks"

It has to do with what it literally says in the owners manual

Autosteer is not intended for use on local roads.

 

[Daniel in SD]

I think it's fairly evident that FSD is much safer on city streets than the current software.

I would strongly disagree with this statement. Beta FSD is capable of making large steering angle changes very quickly. We've seen many examples where it does so incorrectly. I think that makes it more difficult to monitor.

 

[turnem]

I would strongly disagree with this statement. Beta FSD is capable of making large steering angle changes very quickly. We've seen many examples where it does so incorrectly. I think that makes it more difficult to monitor.

The non FSD can make sharp turns as well! It is very quick to jerk me into the wrong lane when it misreads the lane lines.

 

[DanCar]

Probably because the current system explicitly tells you in the owners manual it's not intended to be used on city streets.

That's user error. ...

No, I think that is manufacturer error, when they advertise city features and have a video of AP in the city, but then the owners manual hasn't been updated.

 

[EVNow]

(which is why the idea this was a "total rewrite" is....not terribly accurate... many of the NNs in the FSD beta are the most-current versions of ones that have been in use since 2018, and many more since 2019)

May be they did rewrite those portions and threw out stuff they tried unsuccessfully.

 

[turnem]

No, I think that is manufacturer error, when they advertise city features and have a video of AP in the city, but then the owners manual hasn't been updated.

I agree with you but be prepared for a slew of posts from a guy who’s heels are dug in.

 

[diplomat33]

May be they did rewrite those portions and threw out stuff they tried unsuccessfully.

Yeah. Perhaps, there is some misunderstanding about the rewrite. I don't think the rewrite was necessarily a total redo of every single NN from scratch. We know from Elon that the rewrite involved going from 2.5D to 4D. Perhaps, the rewrite only involved changing the parts needed to make 4D happen. So the rewrite might have only been a redo or changes of some parts, but not everything. I am guessing the rewrite involved adding planning NN, adding video labeling, and changing how the NN for each cameras interconnect in order to make the unified perception possible.

 

[aronth5]

The non FSD can make sharp turns as well! It is very quick to jerk me into the wrong lane when it misreads the lane lines.

I would tend to disagree. I have several sharp S turns that the current FSD aborts even at 20 mph. Similar curves in the beta seem to be handled fairly well.

 

[DanCar]

No, I think that is manufacturer error, when they advertise city features and have a video of AP in the city, but then the owners manual hasn't been updated.

Oops I see an issue. Have to differentiate between FSD and autopilot. For FSD Tesla does advertise its use on city streets. I haven't seen Tesla advertising AP on city streets.

 

[willow_hiller]

Oops I see an issue. Have to differentiate between FSD and autopilot. For FSD Tesla does advertise its use on city streets. I haven't seen Tesla advertising AP on city streets.

This will be an unpopular move, but once autosteer on city streets is a solid feature, I can honestly see Tesla disabling the use of base Autopilot on city streets. It would sell more FSD, and they could say they're doing it for safety.

 

[turnem]

I would tend to disagree. I have several sharp S turns that the current FSD aborts even at 20 mph. Similar curves in the beta seem to be handled fairly well.

So you've never seen the car attempt to jerk you from one lane to another? It happens frequently for me.

Either way - this is argument in futility. Knightshade insists that autopilot use on city streets is explicitly forbidden even though traffic control reactions are currently included in the public software releases.

So apparently we shouldn't be using autopilot on anything except for freeways. Odd that the car allows for us to use autopilot pretty much anywhere but I suppose I'm just being obtuse.

 

[GoSharks]

So you've never seen the car attempt to jerk you from one lane to another? It happens frequently for .

it does not take a large steering input to change lanes. I would in fact call that a minor steering input. A large steering input is something like one you use to take a u turn or switchback.

 

[Daniel in SD]

The non FSD can make sharp turns as well! It is very quick to jerk me into the wrong lane when it misreads the lane lines.

I've never had Autosteer turn the wheel 90 degrees when it's supposed to be driving in a straight line. However I only use it on highways so I don't know. It behaves quite well there and doesn't jerk me between lanes.
If Autosteer behaves like that on city streets I'm not sure why anyone would want to use it there.

 

[Knightshade]

Either way - this is argument in futility. Knightshade insists that autopilot use on city streets is explicitly forbidden

That is untrue of course.

tesla explicitly states it's intended only for use on freeways and other divided roads with controlled access.

It's right there in the owners manual.

You pretending the designer of the system isn't the one telling you this notwithstanding.

So apparently we shouldn't be using autopilot on anything except for freeways. Odd that the car allows for us to use autopilot pretty much anywhere but I suppose I'm just being obtuse.

The car allows you to slam into another car too if you keep it floored. Or drive with your feet.

Neither is a very good idea though.

The car isn't your mom and assumes a reasonably competent driver. Sometimes this is not a valid assumption I guess.