Interesting video. I agree the driver felt uncomfortable but I suspect it's because she is used to how V11 drove. If I were driving I could definitely see me quickly taking that left with some good acceleration which is what V12 did. I think I would have had the same reaction she had but V12 did ok.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
FSD v12.x (end to end AI)
- Thread starter Buckminster
- Start date
-
- Tags
- #fixphantombraking autopilot chill is not chill elon is an ass fix fsd extreme late deceleration fsd does not commit to go fsd does not maintain proper speed fsd hard and late braking fsd jerky driving fsd late deceleration fsd stops too early too quickly fsd stops too far behind line fsd tailgating in city streets
Interesting video. I agree the driver felt uncomfortable but I suspect it's because she is used to how V11 drove. If I were driving I could definitely see me quickly taking that left with some good acceleration which is what V12 did. I think I would have had the same reaction she had but V12 did ok.
Ashok Elluswamy mentioned some recent techniques:
That's why we are working on learning a more general world model that can really just represent arbitrary things. So in this case, what we do is we have a neural network that can be conditioned on the past or other things to predict the future. Obviously everyone has wanted to work on this for ever, and I think with the recent rise of generative models like transformers, diffusion, etc., we finally have a shot at it.
Maybe more details in their "AI Research Scientist, Generative Modeling, Foundation Models, Autopilot AI" role:
- Spearhead the development and training of cutting-edge large generative models, including diffusion models, VAEs, autoregressive models, and GANs.
- Optimize model performance through improved training techniques and methods such as semi-supervised learning and unsupervised pre-training.
- Collaborate with other experts in deep learning, infrastructure, and distributed computing to ensure the efficient and scalable training of large generative models.
Some have mentioned it but here's some thoughtful feedback on v12.2.1's aggressive throttle application. It would seem to be an easy fix especially when the roads are wet etc.
Maybe the plan is to let the other 49 states finally get a crack at it after the team has squeezed every drop of performance out of it.
I don't know why V12.2.1 was temporarily disabled on my car this morning. It's still disabled after I drove more than 1.5 miles. Frustrated, I recalibrated the camera and re-enabled FSD. It was a pain to manually drive and to use only auto steer during 20 miles to recablirate the cameras. Once FSD always FSD.
We went from 2 stacks to "one stack to rule them all" back to 2 stacks? How do they not have millions of miles of highway driving training data since you can gain those much faster than city street driving?
sleepydoc
Well-Known Member
How does v12 do with lane selection when it comes to going straight? One of my biggest reasons for disengagements is FSD’s inability to simple stay in a lane going straight. It inexplicably decides to veer into a turn lane and then panics. It seems like it should be a pretty basic skill/ function for FSD to manage yet it can’t.
We NEVER had "one stack to rule them all" since ASS is/was MIA.
It looks like Tesla is following the same procedure of getting the hard city streets done first and then do the easer limited access highway. I bet this happens fairly fast and wouldn't be surprised if limited access highway was added by 4th qt.EDIT: Just to add highway is pretty good on v11 and I'm glad the effort was to start with the more needed hurky jerky city streets.
stopcrazypp
Well-Known Member
There is much higher risk of death at highway speeds, so they may just be taking a more cautious approach, just like they did in the past with separate stacks. E2E is more unpredictable. As for the miles, while racking up more miles is quicker on the highway, racking up more useful training data (data other than just going in a straight lane) is likely harder than that.
PianoAl
Active Member
I just got back from a 100 mile drive on 55 MPH two lanes and some four lanes. My verdict:
For that situation, V12 was slightly worse than V11. The main problem is that the car often drove too slowly as noted by AI Driver. This is not a phantom braking type thing, and it wasn't dangerous, but it was inconvenient and would have bothered drivers behind me.
Things that were worse
Improvements
The route:
For that situation, V12 was slightly worse than V11. The main problem is that the car often drove too slowly as noted by AI Driver. This is not a phantom braking type thing, and it wasn't dangerous, but it was inconvenient and would have bothered drivers behind me.
Things that were worse
- At one point the car came to an unnecessary stop when going from one road to another, at an intersection that it's handled correctly hundreds of times.
- The auto wipers were not improved or perhaps worse. Recently I've had zero dry wipes, but had a bunch today. That could be related to cleaning the windshield.
- The car seemed to tailgate some other cars, something I have never seen before.
- At one point the car drove between lanes for a while instead of changing lanes. I may have video of that.
- There's a turn where the car has consistently gone out of the lane for forty feet or so, and that has not been fixed.
- It seemed to slow down too much to make a right turn off a road with a 65 MPH limit.
- There are still instances when it should turn on the blinker and it does not.
- You can no longer stop a nag by using the scroll wheels or the shift stalk (this makes it safe, actually).
- I saw the same funky steering wheel dance that AI Driver reported in a destination's parking lot.
- At one point I got a "Take over immediately" red steering wheel for no reason that I could see.
Improvements
- I can start FSD sooner in my driveway than before.
- There's a place where the car would always "lane drift" into the wrong lane, but it didn't do that today.
- There's a turn that the car had always come to a stop and sometimes jerked toward the side of the road, but it handled it correctly today.
The route:
I don't know how but it looks like V12 has more negotiating skills with other drivers.
Neuralink, we have all had monkey chips implanted in our brans while we were asleep. Elon controls and knows all now. Damn that is a conspiracy Elon could get behind.
Unclear if pedestrian/red light issue was a red light run but either way it sounds like something in need of more training.
PianoAl
Active Member
There's a place where v11 always failed on that, and today, v12 went straight (didn't fail).
AlanSubie4Life
Efficiency Obsessed Member
I didn’t see any problems with these throttle applications. I guess on the first one maybe could hear the traction control brake “thunk,” probably caused by suspension bounce on uneven surface. This can happen on dry pavement too. This is why we have traction control. Certainly this sort of profile (the second example shown) should be available on the aggressive setting. There really are very few ways to thread into traffic on busy roads except to punch it, wet or not. With good tires and decent tread, just point the wheels in the direction you want to go, and send it.
Hopefully v12 remains very assertive on the assertive profiles. Lack of assertiveness is a problem with v11 - it’s one contributing reason for continued failures on Chuck’s turn, for example. There were some glimmers of hope in initial videos from Whole Mars though there was also plenty of hesitancy.
As long as they also allow separation of follow distance from assertiveness (they seem clueless here), we’ll be good.
Do you have disk storage to store huge amount of freeway data? And the data is really boring because there are not of many driving patterns from the data.
The techniques for neural network control of vehicles were developed in 1986:
Developed in the late 1980s by a research group at Carnegie Mellon University led by Dean Pomerleau, ALVINN was indeed one of the very first experiments in using neural networks for end-to-end control of an autonomous vehicle.
ALVINN was a groundbreaking project for its time, demonstrating that a neural network could directly learn to control a vehicle based on input from sensors, in this case, images from a front-facing camera. The system was implemented on a modified military Humvee, which was then known as the NAVLAB (Navigation Laboratory) vehicle. ALVINN's neural network was trained to associate camera images with the appropriate steering angles, enabling the vehicle to navigate roads and avoid obstacles without explicit programmed instructions for feature detection or path planning.
nVidia made an end2end network in 2016:
Since then we have had some refinement but mostly just scaling up the compute by a dozen orders of magnudes. That helps:
Eventually we got to the point that Tesla was ready to go end2end. A lot of the work of making FSD v1-v11 has been instrumental in making v12. Gathering the diverse dataset of difficult scenarios. Having the autolabel pipeline to be able to label good and bad drivers. To have the data engine to see where the network is struggling and be able to bring forth millions of examples of the same scenario and iteratively improve the system without humans in the loop.
Tesla's main contribution is their expert understanding of what it takes to accumulate and make use of a massive diverse dataset.
Last edited:
Did some more local testing on v12.2.1 this afternoon with Sport acceleration setting. There seems no big differences on the acceleration and deceleration profile. v12.2.1 did slow down on two dips on one intersection.
The slow down to stop signs seem too early and slow in speed to a stop 0 for me. Some turns seem too slow in speed to take the turn and the subsequent speed up not as quick. But they are all good and smooth. To make the drive more comfortable I can just tap the accelerator to modify the speed in all cases to my taste and that's good.
I think this is a preferred implementation than previous versions of abrupt speed changes of too fast in acceleration in turns and too late to decelerate for stops which did not project a safe drive feel. There was no way of slowing down without disengaging.
The slow down to stop signs seem too early and slow in speed to a stop 0 for me. Some turns seem too slow in speed to take the turn and the subsequent speed up not as quick. But they are all good and smooth. To make the drive more comfortable I can just tap the accelerator to modify the speed in all cases to my taste and that's good.
I think this is a preferred implementation than previous versions of abrupt speed changes of too fast in acceleration in turns and too late to decelerate for stops which did not project a safe drive feel. There was no way of slowing down without disengaging.
He explained the pedestrian/red light issue in another video. Apparently FSD followed another car through the right turn and the light turned red before FSD was at the intersection. The pedestrian began walking across the street as FSD made the right hand turn. So it sounds like another case of a dangerous fail to stop at a red light.
Similar threads
- Article
