-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
FSD v12.x (end to end AI)
- Thread starter Buckminster
- Start date
-
- Tags
- #fixphantombraking autopilot chill is not chill elon is an ass fix fsd extreme late deceleration fsd does not commit to go fsd does not maintain proper speed fsd hard and late braking fsd jerky driving fsd late deceleration fsd stops too early too quickly fsd stops too far behind line fsd tailgating in city streets
Ben W
Chess Grandmaster (Supervised)
I'm taking that into account. For L4 reliability, the car will need to be able to drive at least a million miles on average between necessary interventions (where the car makes a clear mistake that any skilled human driver would avoid). Right now it seems to require such an intervention about once a mile. That's six orders of magnitude improvement needed before it reaches L4, and progress will get more difficult as it gets into the long tail of unusual edge cases. (Many of the current failures are quite common cases, which actually makes it all the more puzzling why they haven't been solved.) Even if the reliability improves by an order of magnitude every two years [exponential improvement], which to me feels like an overly optimistic assessment, that's 12 years before it reaches L4 reliability.
Last edited:
mgs333
Member
FSDtester#1
Craves Electrolytes
I haven't driven V12 yet, but in prior versions, whenever I noticed someone having a worse experience that everyone else isn't, the thread participants usually recommended recalibration of the cameras. Don't know if it will help, but it can't hurt.
AlanSubie4Life
Efficiency Obsessed Member
While I wait for V12 to install, so I can take a worthless spin in the dark with no traffic (reminds me of 10.2 or whatever years ago), my thoughts:
1) It's very very nice to hear a bunch of positive reviews. Of course, we've had this in the past too, but I think overall reviews this time seem somewhat more positive than prior major releases' "first impressions." But I'm extremely jaded & bitter & an inchoate curmudgeon, and until I see v12 able to drive smoothly and stop for slowing traffic ahead without slamming the brakes unnecessarily, or enter a turn lane where it will have to come to a stop with just the right coast speed (a HUGE problem with v11), I'm still skeptical. I'll also be looking at basic residential streets and stop sign performance. These are basic every day things that have to be awesome.
2) Fully agreed on the vast gulf between where we're at and where this needs to be for anything approaching even L3 in actual useful ODDs. That's clear even without driving v12, based on all the videos people have provided.
3) RE your rate of improvement argument: I think the next couple months and the first release or two and how those go will be the real thing of interest. Specifically, it will be good to look at things that Tesla has said they are specifically working on and focusing on, like Chuck's left turn. The next release or two should incorporate a LOT of training for that case (Elon will tell us if it doesn't), so we should certainly see major improvements in success rate there very quickly (way beyond 9 out of 10, perhaps to 99/100 in the next release or two), if this NN training method is to actually be successful (with current hardware). Obviously on Chuck's other test case from his video today it should be very successful as well (rather than 0/3). If this improvement doesn't happen, there will have to be a really convincing explanation - otherwise we would have to assume that training just doesn't work as expected and doesn't actually provide a rapid improvement benefit.
In short, the next beer bet I should certainly lose, otherwise FSD v12 NN end-to-end with current hardware is probably doomed. Cheers.
FSDtester#1
Craves Electrolytes
I am eagerly awaiting your 1st impressions...While I wait for V12 to install, so I can take a worthless spin in the dark with no traffic (reminds me of 10.2 or whatever years ago), my thoughts:
1) It's very very nice to hear a bunch of positive reviews. Of course, we've had this in the past too, but I think overall reviews this time seem somewhat more positive than prior major releases' "first impressions." But I'm extremely jaded & bitter & an inchoate curmudgeon, and until I see v12 able to drive smoothly and stop for slowing traffic ahead without slamming the brakes unnecessarily, or enter a turn lane where it will have to come to a stop with just the right coast speed (a HUGE problem with v11), I'm still skeptical. I'll also be looking at basic residential streets and stop sign performance. These are basic every day things that have to be awesome.
2) Fully agreed on the vast gulf between where we're at and where this needs to be for anything approaching even L3 in actual useful ODDs. That's clear even without driving v12, based on all the videos people have provided.
3) RE your rate of improvement argument: I think the next couple months and the first release or two and how those go will be the real thing of interest. Specifically, it will be good to look at things that Tesla has said they are specifically working on and focusing on, like Chuck's left turn. The next release or two should incorporate a LOT of training for that case (Elon will tell us if it doesn't), so we should certainly see major improvements in success rate there very quickly (way beyond 9 out of 10, perhaps to 99/100 in the next release or two), if this NN training method is to actually be successful (with current hardware). Obviously on Chuck's other test case from his video today it should be very successful as well (rather than 0/3). If this improvement doesn't happen, there will have to be a really convincing explanation - otherwise we would have to assume that training just doesn't work as expected and doesn't actually provide a rapid improvement benefit.
In short, the next beer bet I should certainly lose, otherwise FSD v12 NN end-to-end with current hardware is probably doomed. Cheers.
Ben W
Chess Grandmaster (Supervised)
Good idea, worth a try. The car is a 2017 Model 3, but I would have expected that miscalibrated cameras would have affected v11 as well? V12 is now downloading on my 2022 Model Y as well, we'll see if it's significantly different there too.
Ben W
Chess Grandmaster (Supervised)
No doubt they can improve specific common cases by explicitly bolstering the training set for those cases. The question is whether that approach might inadvertently underweight something else important, or also whether it will help at all the with the long tail of edge cases that don't fit neatly into any category. Their approach seems to be to feed the NN with millions examples of good driving, and ask it to mimic that behavior. But it's not clear whether they have a counterpoint mechanism of feeding it examples of things done _wrong_, and training it _not_ to do those things? (That's often how humans learn most effectively.) For example, it may simply not be "obvious" from the million "good" examples that double-stopping at stop signs is "bad", and this principle does seem to explain a lot of FSD's quirks and failure modes. There's also of course the question of whether the current hardware (HW4) is fundamentally adequate for L4, even with "perfect" software. (Will it need additional camera angles, or more/faster compute, or audio input, or ?)
So, I do expect that they will nail most of the common cases in the next year or two with the current approach. It's the extraordinarily long tail of edge cases that poses the more daunting challenge.
Last edited:
FSDtester#1
Craves Electrolytes
Unfortunately, yes. Fsd branches have been screwed up since it started.
You will have to wait for the V12 branch to catch up to the 2024 branches.
I could be wrong, however it has worked that way for almost 3 years.
Check teslafi and look at what version everyone is coming from.
None are 2024 branches.
Looks like more being added still on teslafi up to 456 from 417 a little while ago.
Last edited:
First drive. Some good. Some bad.
It is smoother. Autospeed needs work. I had a lot of speed variability even with no other traffic around late at night. Also too fast on one neighborhood street but mostly too slow outside of the neighborhood.
I found it biasing a bit to the left side of the lane. Maybe I'll recalibrate cameras tomorrow.
V12 just sat at one green light until I tapped the accelerator. It also made a left turn from a straight through lane. There were no cars anywhere around, so I let it go. But, in the daytime, I'd have to disengage.
V12 slowed for a dark asphalt patch on a neighborhood concrete street. Perhaps saw it as a speed bump. This was repeatable.
No lane drift. Very happy about that.
FSD did successfully navigate 270 deg around a neighborhood micro roundabout that it never could do before. Did it quite smoothly and confidently, though the second time it got a bit close to the outside curb. It was surprisingly jerky on ther similar roundabouts where it was only going 180 deg around. It would come up on them too quickly and hard brake. These roundabouts only have about a six foot diameter island, so they're more like a horizontal speed bump, designed to slow traffic down.
Leaving the neighborhood, the road splits at a median pulling up to a traffic light. V12 tried to go to the left side of the median. FSD has never made that error. There's even double yellow lines. I did this twice with the same error each time. Below is the intersection. I was testing at night with no other cars around.
Going to take some time to get used to this. I've got so many miles on V11 and knew all it's quirks. This is a whole new game. At least it seemed to be fairly consistent on multiple test runs against the same scenario.
It is smoother. Autospeed needs work. I had a lot of speed variability even with no other traffic around late at night. Also too fast on one neighborhood street but mostly too slow outside of the neighborhood.
I found it biasing a bit to the left side of the lane. Maybe I'll recalibrate cameras tomorrow.
V12 just sat at one green light until I tapped the accelerator. It also made a left turn from a straight through lane. There were no cars anywhere around, so I let it go. But, in the daytime, I'd have to disengage.
V12 slowed for a dark asphalt patch on a neighborhood concrete street. Perhaps saw it as a speed bump. This was repeatable.
No lane drift. Very happy about that.
FSD did successfully navigate 270 deg around a neighborhood micro roundabout that it never could do before. Did it quite smoothly and confidently, though the second time it got a bit close to the outside curb. It was surprisingly jerky on ther similar roundabouts where it was only going 180 deg around. It would come up on them too quickly and hard brake. These roundabouts only have about a six foot diameter island, so they're more like a horizontal speed bump, designed to slow traffic down.
Leaving the neighborhood, the road splits at a median pulling up to a traffic light. V12 tried to go to the left side of the median. FSD has never made that error. There's even double yellow lines. I did this twice with the same error each time. Below is the intersection. I was testing at night with no other cars around.
Going to take some time to get used to this. I've got so many miles on V11 and knew all it's quirks. This is a whole new game. At least it seemed to be fairly consistent on multiple test runs against the same scenario.
AlanSubie4Life
Efficiency Obsessed Member
It was ok, but not great, and having zero traffic and darkness makes this report next to useless.
Started with auto set speed on - it doesn't work, unless you like to drive around really slowly. As mentioned, it will speed up if you press the accelerator and sort of stick there, but then over time it will decay back down to some ridiculously slow speed.
Halfway through I switched back to old speed method (I like to set 10% over or so). They broke this too. It doesn't get up to the set speed. You can push the accelerator there too and it sort of works for a while then is broken again.
Mostly speed setting is totally broken, though. Had to think about it, which is not what we want.
1) First intersection (not a stop sign, but advisable to stop since at t-junction traffic across top of the t has right of way), it wasn't way to the left (it needs to turn left) of the street, blocking turning traffic onto my street like v11 did. So that was good. But then it took forever to get going and was jerking the wheel back and forth before it decided to make the turn. Better than v11 but just as slow if not slower.
2) Rolled this. (Have complained about v11 stopping here in the past.) Slow approach, but didn't stop. Good. Way slower than I would do it though.
3) Unprotected left has regressed. Stopped way back from the stop line, took forever to creep forward. There was one vehicle that passed right as it got to the creep limit. It took about 10 seconds after that vehicle (I checked the video) for it to go. Also didn't use the center median entry lane like v11 (regression). Will be garbage with any traffic around.
4) Failed to get in the right lane here to carry straight through. FSD v11 got that right.
5) Failed to slow down properly to enter this turn lane (no traffic behind). Used the brakes significantly. No good. v11 failed here too.
6) Wandered around the parking lot like a blind person. Nearly curbed the wheel on the left going in this entrance. I was ready if it got too close. Stay alert. Useless in parking lots, way too slow; took literally (I reviewed the video) about 20 seconds to negotiate this 4-way stop. Lol. This can't be that hard to get right, can it?
7) Drove on the yellow dots here because it seemed to be shying from a Chevy Blazer or similar vehicle in the right-hand lane (which was not drifting over). So it does seem to move in response to other vehicles, but it is a bit exaggerated. This will be interesting to track with more traffic around.
8) Did way better than v11 entering my neighborhood, but started slowing way too soon. But signaled very early before slowing (good)! In the end I had to add some gas to avoid slowing down traffic behind. V11 would not signal and would also slow down erratically (don't remember what it did exactly, just no good).
Then I did a bunch of residential driving on largely unmarked streets, with stop signs.
Stop signs are terrible. It starts slowing (smoothly) way too soon, approaches the stop sign too slowly, stops at least 5 feet behind the stop line, then cautiously advances and eventually goes. Broken. On one stop sign (stop line obvious of course) which was obscured by vegetation, it didn't see it until the last second, and then it had to slam on the brakes last second (so it could stop 5 feet in front of the stop line, then advance slowly). Bad! This weird excess early slowing behavior does not only happen at stop signs. It seems to affect most turns (the one entering my neighborhood, left turn lanes controlled by lights, etc.). Broken.
Steering is jerky and uncertain on turns after stop signs on these residential streets.
As mentioned speed control is completely broken. On one downhill stop sign it failed to slow sufficiently soon and used excessive brakes. I had turned off auto speed and had set the speed to 38 (in a 25) though it was not doing that (it is broken as mentioned and does not go to the set speed).
It did well on the second left-turn lane test case. It slowed before entering, though after it entered it was too slow to advance to the light (clear pattern developing). But good that it did not use the brakes and slowed while it was in the through lane (required, to not enter turn lane at excessive speed, though if traffic is close behind I would use my brakes without slowing much in the through lane). v11 would NEVER slow before entering the turn lane. It was brake-slamarama garbage.
But way more exciting than anything else is FSD did not run over an opossum. Now, I would have saved the opossum, even if FSD had wanted to kill it. But v12.3 did not want to kill it. This is a good sign for AI. Elon might be wrong about the AI. It may be good.
If you watch carefully, you'll see that FSD reaction time was relatively good. It matched mine (watch my foot, and hand, and watch the regen bar). Now, it was night, and my night vision is not that good - so I was actually kind of slow - took me about 1-1.5 seconds to react (the possum is much more easily seen in the video than through my eyes, however). I probably would have been faster if I had been in control of the accelerator, though. (Much safer to drive manually as long as you're paying attention!). I think I was also looking at the map to select the desired route (note I had only one hand on the wheel initially since I had been using the screen
).In any case it stopped for the opossum, which was awesome.
Overall summary: I would say that v12.3 seems worse than v11 for my particular route tonight (as mentioned it's nearly a useless test case). But I can't extrapolate from this drive to my commute - it's a much different set of considerations. For the commute, the key criteria is stopping for lights and vehicles in front smoothly and consistently (something v11 cannot do, but something I could not test tonight). It does seem smoother and less robotic but I definitely would not describe it as smooth or not robotic. We'll see. I can see the promise, if they can quickly address the obvious problems.
The good news is that even if it sucks and is unusable for my commute for a while, that's ok, since v11 really wasn't a joy to use, or really that useful. So nothing really lost.
Last edited:
What? My car is Driverless all the time. Minnie Driver has never been in my car.
My initial report on v12.3:
- First drive was late at night with very little traffic, which is part of my daily driving anyway - so a valid comparison.
- Overall I'm happy. No disengagements, no observed regressions or new problems (but see discussion of speed control below).
- My evaluation of smoothness versus v11 is mixed:
- There was no jerkiness, and lines through turns were probably better.
- Contrary to what some people have reported, I think v11 in average mode had smoother stops and starts at traffic lights.
- I switched from Average to Chill on V12 late in the drive, and I think that was more comfortable for me. I need more experience in normal traffic to decide which is really better.
- First results indicate that my big problem of v11 refusing to be in the correct lane, more than ~1,000 ft ahead of certain turns, is fixed. Yay! I hope this holds true in daytime traffic.
- OTOH I found one or two places where the v11 was earlier=better at self-selecting the correct lane well before a turn. This is not a big deal as long as v12 accepts my stalk-initiated lane request without a subsequent fight.
- Other notable improvements:
- v12 fixed a spot where v11 always changed lanes through a traffic light intersection.
- v12 fixed a spot where there is a traffic light on the left only for U-turns (a "Michigan Left Turn" path). v11 always got spooked and required accelerator input to push through the main traffic lanes without slowing/stopping.
- v12 correctly avoided moving into a new right lane where the road widens from 2 to 3 lanes.
- v12 did a better job of slowing comfortably before a dedicated right turn/merge path, where v11 approached it a bit too fast.
- v12 performed a U-turn as called for by the Nav, that v11 never would do. As reported by others, the U-turn speed was too slow and over-cautious, but not ridiculously so. Needs more testing to see how it responds to threats from cars wanting to turn right into my U-turn right-of-way (and whether it's smart enough to yield at other intersections marked "U-turn YIELD to Right Turn").
- I tried the Auto speed offset feature at first, but it immediately ramped up too fast (for me) on a semi-residential road, and then seemed somewhat inconsistent on suburban multi-lane roads (45 mph posted).
- Eventually I switched to non-auto, zero speed offset. This was more predictable, and more like v11 except that the response to speed scroll wheel input, whether up or down, was extremely laggy and sluggish.
- With v11, I could quickly slow the car down with the wheel anytime I felt unsure about the traffic situation just ahead. And I could override the cautiously-set zero speed offset to keep up with prevailing traffic, knowing that I could revert quickly at any time.
- With v12, I feel that I've lost this control in a practical sense. If I want to go faster, it will eventually comply but it seems to take a long time. If I want to go slower "right now" without disengaging by hitting the brake, I really can't do it with v12. I'd like to see the v11-like scroll wheel responsiveness come back to V12.
Similar threads
- Article
