FSD v12.x (end to end AI)

[TK211X]

Another small drip (can't really call it a push) of about 200 on TeslaFi.

View attachment 1041751

Supposedly the testers are validating a new version. Why even bother with 12.3.5 if it’s a clear regression.

 

[Daniel in SD]

I love the euphemism of validating. At this point it's more likely chasing pesky bugs and prepping for an exorcism.

I wonder if previous releases were “validated” and found to have an acceptable failure rate?
When you have a 10-20% failure rate it doesn’t seem like “validating” would take very long.
I’m curious what the actual process is here. Are there a bunch of inputs to the model that they can tweak?
Or are they collecting training data?

 

[EVNow]

There are bugs, but the people from the company at the earnings call are convinced that said bugs are fixable. That's a new statement.

Thats not what they said.

"Yeah. In terms of scaling loss, people in the community generally talk about model scaling loss where they increase the model size a lot and then their corresponding gains in performance, but we have also figured out scaling loss and other access in addition to the model side scaling, making also data scaling. You can increase the amount of data you use to train the neural network and that also gives similar gains and you can also scale up by training compute, you can train it for much longer and one more GPUs or more dojo nodes that also gives better performance, and you can also have architecture scaling where you count with better architectures for the same amount of compute produce better results. So, a combination of model size scaling, data scaling, training compute scaling and the architecture scaling, we can basically extrapolate, OK, with the continue scaling based at this ratio, we can perfect big future performance."

In other words, they think based on how much they can improve the model by putting more data - they can extrapolate.

ER Call Transcript

 

[Tronguy]

Thats not what they said.

"Yeah. In terms of scaling loss, people in the community generally talk about model scaling loss where they increase the model size a lot and then their corresponding gains in performance, but we have also figured out scaling loss and other access in addition to the model side scaling, making also data scaling. You can increase the amount of data you use to train the neural network and that also gives similar gains and you can also scale up by training compute, you can train it for much longer and one more GPUs or more dojo nodes that also gives better performance, and you can also have architecture scaling where you count with better architectures for the same amount of compute produce better results. So, a combination of model size scaling, data scaling, training compute scaling and the architecture scaling, we can basically extrapolate, OK, with the continue scaling based at this ratio, we can perfect big future performance."

In other words, they think based on how much they can improve the model by putting more data - they can extrapolate.

ER Call Transcript

Musk's words, from the same transcript:

Elon Musk -- Chief Executive Officer and Product Architect

Yeah. And we do have some insight into how good the things will be in like, let's say, three or four months because we have advanced models that our far more capable than what is in the car, but have some issues with them that we need to fix. So, they are there'll be a step change improvement in the capabilities of the car, but it will have some quirks that are -- that need to be addressed in order to release it. As Ashok was saying, we have to be very careful in what we release the fleet or to customers in general.

So, if we look at, say, 12.4 and 12.5, which are really -- could arguably even be Version 13, Version 14 because it's pretty close to a total retrain of the neural nets in each case are substantially different. So, we have good insight into where the model is, how well the car will perform in, say, three or four months.

------------------
That's not "aggressive guessing". That's saying that they've got advanced models right now that's better than the 12.3.x that we're driving around upon.
They've qot "quirks" to solve. But they're not inventing something new. They've got it now and they're beating on them.

This isn't "future extrapolation" in the nebulous future. Which is what people usually complain about with Musk.

 

[EVNow]

You are confusing the two things.

So, we have good insight into where the model is, how well the car will perform in, say, three or four months.

The above is not the same as what you claimed.

Tesla is currently testing candidates for Robotaxi. There are bugs, but the people from the company at the earnings call are convinced that said bugs are fixable.

All Elon is saying is that they know 3 to 6 months out what they have got. He didn't claim what they have got in 3 to 6 months is "robotaxi" candidates. You don't go from 1 disengagement in 20 miles to 1 disengagement in 10,000 miles in 3 to 6 months. I'll be happy if they are at 1 disengagement in 30 miles in 3 to 6 months

Let me put it out for anyone wondering about the "CyberTaxi".

CyberTaxi is a *hardware* reveal. They don't have any plants to build that yet. They don't even know where they will build it - just that it will be on this totally new process and will take a couple of years to start building. They hope, may be, that by then they will have something like a robotaxi capable software. My guess is
- They will start the "CyberTaxi" with real safety drivers in limited geographic areas like all others did in 3 to 5 years
- Once they have enough miles to show they are safe, they will apply for real robotaxi permits and get them

All assuming they make real progress on that FSD stuff - not just "extrapolating".

 

[AlanSubie4Life]

I wonder if previous releases were “validated” and found to have an acceptable failure rate?
When you have a 10-20% failure rate it doesn’t seem like “validating” would take very long.
I’m curious what the actual process is here. Are there a bunch of inputs to the model that they can tweak?
Or are they collecting training data?

I think they are just happy to get paid to repeatedly drive the corner.

And the engineers back home don’t even bother checking whether it works. There’s not much point because if it is broken they don’t have a way to fix it.

Clearly it fails all the time in validation.

Gonna go out on a limb and say this means 12.4 will also not work. It is mind-boggling though; it seems like it should just work if they actually try to make it work and that is an actual requirement.

 

[Knightshade]

Gonna go out on a limb and say this means 12.4 will also not work. It is mind-boggling though; it seems like it should just work if they actually try to make it work and that is an actual requirement.

Almost like the cameras visibility isn't sufficient for the situation or something, but they don't want to admit they need to add a better side-view camera forward of the B-pillar or something!

 

[aronth5]

Everyday I use FSD for every drive and it's been weeks since I had a critical safety issue. Certainly couldn't say that on V11 which had critical safety issues all the time.

• My most common disengagements now are still potholes and police/road crews using hand gestures since there is so much road work beginning now that winter is over.
• Speed control still needs improvement but since I'm usually driving with other cars nearby FSD does a reasonable job of matching their speeds.
• Like everyone else lane change selection needs work.
• Most annoying problem- FSD is too timid at intersections requiring accelerator pushes. Super annoying.

Overall- if I never disengage FSD would make the destination every time. FSD might have to reroute when it screws up lane changes but that is my experience. Doubt FSD would be this successful if I drove more in Boston though.

 

[AlanSubie4Life]

Almost like the cameras visibility isn't sufficient for the situation or something, but they don't want to admit they need to add a better side-view camera forward of the B-pillar or something!

The thing is this doesn’t even seem to be the problem right now.

As far as I can tell the car can see cars more than six seconds out at 45-55mph or so. This should be plenty for basically everything but the most busy scenarios (for this turn) as long as it hits 25-30mph by the apex.

But it just fails miserably in different ways from release to release. Just freezing in the middle of the road does not seem like a camera problem. Also seems like something that would show up in simulation. If it doesn’t I would propose fixing their simulation.

• Most annoying problem- FSD is too timid at intersections requiring accelerator pushes. Super annoying.

Yep. No good. Manifests in a few ways.

 

[rlsd]

FSD needs to have an icon to clearly indicate whether destination is set or not. Also it needs to remind the user if destination is not set when FSD is engaged.

FSD was innocently killed by me 4 times in less than 5 minutes.

I drove my car from a grocery parking to lane 1 of a 3 lane street then engaged FSD.
Suddenly the car jumped to lane 2. I disengaged and moved the car back to lane 1 so that it could make left turn to my neighborhood in .3 mile.
I engaged FSD again but the car jumped to lane 2 again in less than a minute. I disengaged and moved the car back to lane 1, re-engaged FSD and manually changed to the designated left turn lane. While waiting for left turn green light, FSD moved the wheel to the right and jumped back to lane 1. I thought FSD was crazy. I disengaged again and made the left turn when green light was on. Then I re-engaged FSD and I expected the car to make right turn at the next block. But FSD accelerated and skipped the right turn and went straight. I disgenaged and realized that I did not set destination for FSD.

 

[Knightshade]

FSD needs to have an icon to clearly indicate whether destination is set or not.

You mean like the big box in the upper left of the map showing routing, or the one in the bottom left of the map showing destination and ETA?


BTW another reason no FSD on current (hw3 anyway) cars- today just sitting at a red light (FSD on) I got the red steering wheel TAKE OVER IMMEDIATELY message because one of the nodes on the FSD computer crashed.... Since they've long since run out of single-node compute there's no redundancy.

If I hadn't been stopped, and there was no driver, well....

 

[ArtK]

I have FSD, and paid a total of $6000 for it — $4000 for EAP a couple of years ago and $2000 a few days ago when the upgrade price was announced. Even though FSD, viewed objectively, drives like an impulsive 16 year-old showing off for friends (endlessly changing lanes for no real reason; slowing up for no real reason; getting easily confused at lane merges) I kind of enjoy having it as my co-pilot, always keeping me on my toes to avoid sudden disaster. And if my focus wanders, FSP is really good at noticing an abrupt slow-down in the flow of traffic. I kind of enjoy constantly competing with it. It’s kind of interesting and almost fun.

 

[gottagofast]

Everyday I use FSD for every drive and it's been weeks since I had a critical safety issue. Certainly couldn't say that on V11 which had critical safety issues all the time.

• My most common disengagements now are still potholes and police/road crews using hand gestures since there is so much road work beginning now that winter is over.
• Speed control still needs improvement but since I'm usually driving with other cars nearby FSD does a reasonable job of matching their speeds.
• Like everyone else lane change selection needs work.
• Most annoying problem- FSD is too timid at intersections requiring accelerator pushes. Super annoying.

Overall- if I never disengage FSD would make the destination every time. FSD might have to reroute when it screws up lane changes but that is my experience. Doubt FSD would be this successful if I drove more in Boston though.

Mine is a little too aggressive on lane changes even in chill, another car will be coming up on me pretty fast in the other lane and fsd just decides it wants to change lanes

Then the other car that was coming fast is forced to slow when my car changes lanes in front of them

Yes it had time and room to change lanes but I don’t like making other people have to hit their brakes

 

[Ctuna]

I have FSD, and paid a total of $6000 for it — $4000 for EAP a couple of years ago and $2000 a few days ago when the upgrade price was announced. Even though FSD, viewed objectively, drives like an impulsive 16 year-old showing off for friends (endlessly changing lanes for no real reason; slowing up for no real reason; getting easily confused at lane merges) I kind of enjoy having it as my co-pilot, always keeping me on my toes to avoid sudden disaster. And if my focus wanders, FSP is really good at noticing an abrupt slow-down in the flow of traffic. I kind of enjoy constantly competing with it. It’s kind of interesting and almost fun.

Is there a discussion forum for FSD. I would like to see a thread with peoples experiences. I have been on the trial and it is both amazing an confusing . It certainly does not drive that smoothly . It will tend to make turn after sneaking out slowly into the lane then almost floor it. Goes exceedingly slow in some places where it seems to be confused. Stays to close to Semi's on the highway . Picks the wrong speed sometimes mostly too slow till it see's the next sign.
I occasionally get it painting a parked car Red as your passing it or if there is turn coming up and you are headed toward the parked car even though there is a turn it paints it Red and beeps.
Anybody want to start a discussion thread on there experiences.
It would be nice if you could store a scenic route you want to take or
a preferred route you use in town.

 

[JHCCAZ]

I pressed them on it (I actually had to approve a $255 estimate for them to do the diagnosis at all), and when they took off the camera housing they discovered a thick layer of chemical residue on the inner glass and camera lenses. They cleaned it, apologized, comped the diagnostic fee ("Goodwill")

That's great, I'm very glad you got it resolved. As I said in my earlier posts, I think there are quite a few cars in the fleet with this problem, to a greater or lesser degree.

But I also said that Tesla must be well aware of it, so I wasn't too happy to read the next part:

and expressed disbelief that this isn't part of the standard annual maintenance checklist, at least for FSD-enabled cars. Now the problem appears to be completely fixed.

IMO Tesla needs to take this very seriously and do the following things, some can be accomplished now and some for the future:

1. Elevate the issue above the level it already has which is some kind of service bulletin note.
   1. Include it in training materials and flag it using whatever other internal communication methods would be appropriate.
   2. It really doesn't matter if the percentage of seriously affected vehicles is low - the problem is very serious in those vehicles, completely distorting the primary camera view with low sun or bright nighttime lights.
2. Assign a team member to work the problem by finding and or creating a set of training clips that teach the AI to self diagnose this problem. The kind of smearing glare that it creates is not hard to teach. Then the car can notify the driver and notify the factory that cleaning service is required.
3. Create a service tool that every service center should have, to validate the cleanliness of windshield and lenses for the front cameras. I'm thinking a kind of box with a pattern of bright LEDs and perhaps a printed pattern also inside. The box is designed to mate against the upper windshield, and a standard test software routine runs to validate cleanliness, calibration and basic video quality. If it doesn't pass, it requires a cleaning or camera service/replacement.
4. As I said before, redesign the camera housing so that it's easily removable for owner DIY cleaning or service-tech cleaning. A simple squeeze clip arrangement instead of the present situation that requires proper tools and technique just to get a cover off, then unplug connectors and Dismount the mirror and loosen some screws to jiggle the cover off, it's way too hard to maintain the cleanliness right now, and obviously the materials used are prone to outgassing and hazy film deposits.

Everyone should note that some users here report vastly inferior results with the same software that others are quite happy with. I proceed that a fair number of the complaints come from owners of relatively older cars. Of course some of this is down to expectations, personality and so on, but I believe some of it maybe due to hazy/glary front cameras, condensation on pillar cameras, light leaks in repeater cameras and inevitable accumulated gunk on rear cameras.

It's a shame when mundane issues like these get in the way of (what I consider) a remarkable software and data infrastructure effort by Tesla. I hope they consider these suggestions that could improve performance, safety and owner satisfaction.

 

[fasteddie7]

Another small drip (can't really call it a push) of about 200 on TeslaFi.

View attachment 1041751

One of my three cars got it yesterday. This is the first time all three didn’t get the update at the same time.

 

[BBTX]

Couldn't they train a very small net to detect for the haze and other such camera issues and then report the issue on screen and recommend servicing?

 

[BBTX]

FWIW, I'm still very positive on 12.x so far. I've been letting it drive a ton. I can nitpick lots of minor issues but bottom line is it's safe, it hasn't curbed my wheels, and it will eventually get me there. The closest thing I've seen to wheel damage is it ignoring common potholes on the edge of a right hand turn and letting the rear wheel plonk through them.

 

[Dewg]

Almost like the cameras visibility isn't sufficient for the situation or something, but they don't want to admit they need to add a better side-view camera forward of the B-pillar or something!

If the Tesla AP page is at all accurate, at least for HW3, the B-Pillars can only see 80m (262 feet), and the repeaters can see 100m (328 feet).

Autopilot

All new Tesla vehicles are equipped with the hardware needed for Autopilot and Full Self-Driving capability features, designed to improve functionality over time.

www.tesla.com

 

[AlanSubie4Life]

If the Tesla AP page is at all accurate, at least for HW3, the B-Pillars can only see 80m (262 feet), and the repeaters can see 100m (328 feet).

Autopilot

All new Tesla vehicles are equipped with the hardware needed for Autopilot and Full Self-Driving capability features, designed to improve functionality over time.

www.tesla.com

View attachment 1041855

As with most things on that page, don’t believe it!

That would only be 4 seconds at 45mph and it is very clear the car can see considerably further than that (I would guess it can see about six seconds at that speed at a minimum).

With what reliability it is hard to say of course!

None of these numbers are good compared to humans, but it may be enough to allow for a very high success rate on this turn.