Wiki MASTER THREAD: Actual FSD Beta downloads and experiences

[EVNow]

How can AP work better than production version when that stack remains production? Single stack doesn't exist yet, does it?

The new AP in 10.4 might also be "map aware" like FSD - so it can figure out how to go straight when the lane splits etc. I'll try AP in complex intersections where production AP always failed ... that would confirm my hypothesis.

 

[EVNow]

But according to Musk radar just adds noise and no improvement in signal compared to what they get using their pseudo-lidar with vision only.

I wouldn't trust a biased opinion that has billions at stake because of a radar chip shortage.

I think radar was adding noise - the data Karpathy showed on AI day wasn't purely made up.

But they may have jumped the gun and introduced VO early because of chip shortage. If there was no chip shortage - may be we would have radar in new cars for another year until VO was in parity with Radar + Vision.

 

[old pilot]

Clear day, dry roads, on chill mode. I'm learning how to better interface with the FSD and am more aware of what does and doesn't happen when engaging and disengaging and how and when to use the accelerator. Thanks to all who, over the past month, have posted helpful tips and observations. I have found that by taking more control of the speed of the car ( with right thumb wheel and accelerator) operating the AP has become more manageable. The car is making most (not all) turns safely (some jerking). Most impressive was a protected left with two lanes to chose (using navigation) got into the correct lane (left) knowing that in a short distance after the turn it would need to be in the left lane to continue on course. Made lane changes to position itself for other left turns. On one protected left (with opposing traffic also turning left and the opposing lanes slightly offset, not a true + crossroads intersection) disengaged because I was not sure what to expect. I'm not sure if what I experience is what others refer to as "phantom braking" ( seemed to happen more with shadows on the road) for me its more of a unexplained speed reduction (not a hard stop) so maybe someone can add some clarity to that for me.

 

[Cosmacelf]

So now that we've got a CNN review from a "skittish" first time Tesla driver (!), I thought I'd weigh in based on my own experiences.

First, is it possible to do a drive with no interventions? Yes. But there are a number of problems, and this post will focus on those.

Camera Suite Needs Upgrading

My neighborhood has a lot of 2 lane roads with funky intersections, making left turns challenging even for human drivers. Here's an intersection that gives FSD problems.




You are turning left as shown by the yellow arrow. Traffic coming from the left is obscured by a bend in the road. FSD actually recognizes this since it will creep into the intersection to try to get better vision. Unfortunately, it creeps so far that traffic coming from the left actually has to swerve to miss clipping the front of my car. Chuck Cook did a good video analysis of how the Tesla B pillar cameras (the only ones that give you a left/right 90 degree view) aren't up to the task (

).

IMHO, Tesla needs better left/right 90 degree vision. Maybe A pillar cameras, or bumper cameras as Chuck simulated, or both. You may not realize it, but there are many times when you are driving when you move your entire body to get a better look when turning in an intersection. You do it so automatically that you don't even realize it, but good human drivers do move themselves to get clear vision. Tesla's B pillar cameras just aren't up to the task.

Perception/Planner Needs Smoothing

This is something that has bothered me since the very first AP v1.0. The system that creates a 3-d view of the world has very little to no temporal smoothing. You see cars wink in and out of existence in the visualizer, or shift sideways by a foot and then shift right back. While this is mildly annoying to see in the visualizer, it has real world negative consequences when the planner does the same.

You can see the effects in the CNN review (Tesla owner lends FSD Beta to inexperienced operator from CNN--to chaotic results) when the car quickly turns the steering wheel towards the path of an oncoming UPS truck in the next lane. I am guessing here that the planner thought the UPS truck was moving slow enough or stopped that it could partially move into the UPS truck lane to dodge the scooter (rather than just slow down). This points to an inadequacy of the planner (humans would have assumed the UPS truck isn't stopped) and vision system (the vision system isn't as accurate as radar in determining vehicle speeds).

But beyond those problems, the system reacts too quickly. Isn't that a good thing, you ask? No, not when the result is moving into an oncoming vehicle. Now the CNN reviewer was skittish, so he immediately took over control. From personal experience, I am pretty sure the car would have realized its mistake a split second later and turned back and slowed down.

But this behavior gives drivers heart attacks. It isn't uncommon in beta 10.4 for the steering wheel to whip side to side or jitter at like 10 hz or something. Both the perception system and path planner could have a dose of real world physics inserted into their code. Cars cannot jump left a foot. Nothing useful comes from jittering the steering wheel back and forth at 10 hz. When something unexpected occurs, take a moment to gather more data before reaching a hasty (and often wrong) conclusion.

Whoa - Dual Left Turn Lanes

Here's an example of a major intersection where there are dual side by side left hand turn lanes.




In a recent FSD drive, my car was in the left most position in this picture waiting to turn left. When making the turn, the car beside me was slow starting their turn, and my car crossed lanes while making the turn and ended up in the leftmost lane heading away from the view in this picture.

I don't know what would have happened if the driver beside me had been keeping up - would my car have noticed the car beside me and turned into the correct lane? Probably, I hope. Nonetheless, this is simply bad driving and no doubt freaked the guy beside me out since I went into his lane in the middle of the intersection.

Another time, to avoid some traffic ahead of me across an intersection, my car signaled to change lanes and would have changed lanes in the middle of the intersection. Again, it would have worked since there wasn't a car beside me, but this is either illegal or just horrible driving (never change lanes in the middle of an intersection).

Right Turn Lanes

This same intersection, like a lot of large intersections, has a mini right turn "lane" where cars, assuming drivers going straight give them space, go into to turn right.




FSD does not recognize these lanes and does not use them. This is does not work well in practice and causes all sorts of problems.

Mode Confusion

There are two software coded behaviors of FSD which are unsafe, and one of them has been there forever. I am appalled that Tesla doesn't fix this.

When you take over control by grabbing the steering wheel, FSD turns off and tells you that it turned off by a playing a chime. BUT IT KEEPS TACC ON. So now you naturally think you've got full control of the car, and by the way, you've taken control because of something that demands your full attention ahead of you outside the car. So you drive for a while and eventually you ease off the go pedal to take a corner, BUT TACC IS STILL ON and keeps the car driving at the full speed limit through the corner! This is massively unsafe and Tesla needed to fix this years ago but continues to not fix this.

Here's a new bad bug. Try turning on FSD with NO ROUTE SET in the Nav. You'd think the system wouldn't allow you to turn it on since where is the car going to go? The car actually does just sit there, so thinking that you've a route set (I'm going home afterall), you give the car a press of the go pedal. The car now starts driving by itself. It really does. I haven't fully characterized it, but it seems to follow other cars in front of it, turning when they do, etc. Given the driver thinks the car should be going somewhere else typically, this causes a bit of angst. Anyways, FSD should give you an error, not allow it to be turned on with no route set.

Hugs Center Lane

Way too often, on a 2 lane roads, the car will hug the double yellow lane divider, sometimes driving for 100 feet over the cat eye reflectors in the road (bump, bump, bump). So yeah, not technically over the yellow lane divider, but on narrow roads, this is actually unsafe. I have no idea why FSD has a left of lane bias, but it does. Of note, the car knows it is driving right next to the center road lane divider since the visualizer shows it doing so, so the car knows it is doing this and yet doesn't correct to re-center in the lane when on long sweeping turns.

Conclusion

In its current state, I only use FSD when I decide to spend the time to give Tesla feedback. I don't use it for regular driving - it just doesn't work well enough for that. Contrast that with freeway autopilot which I do use routinely. I have a 2019 Model X with radar, so freeway autopilot, which uses the older software stack, still works great.

Based on all the "no intervention" videos I saw around, I thought FSD was further ahead than it is.

Tesla still has a bunch of just routine behaviors and perception issues it needs to fix. Most concerning is that I suspect it will need a new camera suite to work properly.

 

[TIppy]

I don't know if it's enabled yet, but Musk said if no route is set the car would either navigate to work or home depending on time of day, or it would check your calendar for a location.

 

[Phlier]

I'm not sure if what I experience is what others refer to as "phantom braking" ( seemed to happen more with shadows on the road) for me its more of a unexplained speed reduction (not a hard stop) so maybe someone can add some clarity to that for me.

Yup, it would appear that as Tesla attempts to solve the Phantom Braking problem, they have managed a partial fix with Phantom Slowing.

In 10.2, I would get hard phantom braking consistently in certain places. Then in 10.3.1 (and now also in 10.4), most of the places that resulted in phantom braking now result in phantom slowing; rather than hitting the brakes hard, the car just moderately slows down, instead.

That's not to say that I don't still get the odd phantom braking, with its seat belt locking shenanigans... that still does happen. It's just not as frequent, and more commonly I get phantom slowing instead.

Shadows on roads have long been a problem for Tesla, and has been well documented here for years. Yet in my experience, I have yet to see a shadow on the road when I get phantom braking or slowing. So there's a lot more going on here than just shadows.

And it appears to be a seriously difficult problem to solve.

 

[EVNow]

I don't know if it's enabled yet, but Musk said if no route is set the car would either navigate to work or home depending on time of day, or it would check your calendar for a location.

You can set it as an option. In the morning when I leave, FSD picks "work" as the destination by default.

 

[Phlier]

So now that we've got a CNN review from a "skittish" first time Tesla driver (!), I thought I'd weigh in based on my own experiences.

First, is it possible to do a drive with no interventions? Yes. But there are a number of problems, and this post will focus on those.

Camera Suite Needs Upgrading

My neighborhood has a lot of 2 lane roads with funky intersections, making left turns challenging even for human drivers. Here's an intersection that gives FSD problems.




You are turning left as shown by the yellow arrow. Traffic coming from the left is obscured by a bend in the road. FSD actually recognizes this since it will creep into the intersection to try to get better vision. Unfortunately, it creeps so far that traffic coming from the left actually has to swerve to miss clipping the front of my car. Chuck Cook did a good video analysis of how the Tesla B pillar cameras (the only ones that give you a left/right 90 degree view) aren't up to the task (

).

IMHO, Tesla needs better left/right 90 degree vision. Maybe A pillar cameras, or bumper cameras as Chuck simulated, or both. You may not realize it, but there are many times when you are driving when you move your entire body to get a better look when turning in an intersection. You do it so automatically that you don't even realize it, but good human drivers do move themselves to get clear vision. Tesla's B pillar cameras just aren't up to the task.

Perception/Planner Needs Smoothing

This is something that has bothered me since the very first AP v1.0. The system that creates a 3-d view of the world has very little to no temporal smoothing. You see cars wink in and out of existence in the visualizer, or shift sideways by a foot and then shift right back. While this is mildly annoying to see in the visualizer, it has real world negative consequences when the planner does the same.

You can see the effects in the CNN review (Tesla owner lends FSD Beta to inexperienced operator from CNN--to chaotic results) when the car quickly turns the steering wheel towards the path of an oncoming UPS truck in the next lane. I am guessing here that the planner thought the UPS truck was moving slow enough or stopped that it could partially move into the UPS truck lane to dodge the scooter (rather than just slow down). This points to an inadequacy of the planner (humans would have assumed the UPS truck isn't stopped) and vision system (the vision system isn't as accurate as radar in determining vehicle speeds).

But beyond those problems, the system reacts too quickly. Isn't that a good thing, you ask? No, not when the result is moving into an oncoming vehicle. Now the CNN reviewer was skittish, so he immediately took over control. From personal experience, I am pretty sure the car would have realized its mistake a split second later and turned back and slowed down.

But this behavior gives drivers heart attacks. It isn't uncommon in beta 10.4 for the steering wheel to whip side to side or jitter at like 10 hz or something. Both the perception system and path planner could have a dose of real world physics inserted into their code. Cars cannot jump left a foot. Nothing useful comes from jittering the steering wheel back and forth at 10 hz. When something unexpected occurs, take a moment to gather more data before reaching a hasty (and often wrong) conclusion.

Whoa - Dual Left Turn Lanes

Here's an example of a major intersection where there are dual side by side left hand turn lanes.




In a recent FSD drive, my car was in the left most position in this picture waiting to turn left. When making the turn, the car beside me was slow starting their turn, and my car crossed lanes while making the turn and ended up in the leftmost lane heading away from the view in this picture.

I don't know what would have happened if the driver beside me had been keeping up - would my car have noticed the car beside me and turned into the correct lane? Probably, I hope. Nonetheless, this is simply bad driving and no doubt freaked the guy beside me out since I went into his lane in the middle of the intersection.

Another time, to avoid some traffic ahead of me across an intersection, my car signaled to change lanes and would have changed lanes in the middle of the intersection. Again, it would have worked since there wasn't a car beside me, but this is either illegal or just horrible driving (never change lanes in the middle of an intersection).

Right Turn Lanes

This same intersection, like a lot of large intersections, has a mini right turn "lane" where cars, assuming drivers going straight give them space, go into to turn right.




FSD does not recognize these lanes and does not use them. This is does not work well in practice and causes all sorts of problems.

Mode Confusion

There are two software coded behaviors of FSD which are unsafe, and one of them has been there forever. I am appalled that Tesla doesn't fix this.

When you take over control by grabbing the steering wheel, FSD turns off and tells you that it turned off by a playing a chime. BUT IT KEEPS TACC ON. So now you naturally think you've got full control of the car, and by the way, you've taken control because of something that demands your full attention ahead of you outside the car. So you drive for a while and eventually you ease off the go pedal to take a corner, BUT TACC IS STILL ON and keeps the car driving at the full speed limit through the corner! This is massively unsafe and Tesla needed to fix this years ago but continues to not fix this.

Here's a new bad bug. Try turning on FSD with NO ROUTE SET in the Nav. You'd think the system wouldn't allow you to turn it on since where is the car going to go? The car actually does just sit there, so thinking that you've a route set (I'm going home afterall), you give the car a press of the go pedal. The car now starts driving by itself. It really does. I haven't fully characterized it, but it seems to follow other cars in front of it, turning when they do, etc. Given the driver thinks the car should be going somewhere else typically, this causes a bit of angst. Anyways, FSD should give you an error, not allow it to be turned on with no route set.

Hugs Center Lane

Way too often, on a 2 lane roads, the car will hug the double yellow lane divider, sometimes driving for 100 feet over the cat eye reflectors in the road (bump, bump, bump). So yeah, not technically over the yellow lane divider, but on narrow roads, this is actually unsafe. I have no idea why FSD has a left of lane bias, but it does. Of note, the car knows it is driving right next to the center road lane divider since the visualizer shows it doing so, so the car knows it is doing this and yet doesn't correct to re-center in the lane when on long sweeping turns.

Conclusion

In its current state, I only use FSD when I decide to spend the time to give Tesla feedback. I don't use it for regular driving - it just doesn't work well enough for that. Contrast that with freeway autopilot which I do use routinely. I have a 2019 Model X with radar, so freeway autopilot, which uses the older software stack, still works great.

Based on all the "no intervention" videos I saw around, I thought FSD was further ahead than it is.

Tesla still has a bunch of just routine behaviors and perception issues it needs to fix. Most concerning is that I suspect it will need a new camera suite to work properly.

That was a fantastic post, thanks for sharing!

 

[Phlier]

Based on all the "no intervention" videos I saw around, I thought FSD was further ahead than it is.

That is exactly what I thought when I went out on my first FSD drive.

I'm convinced that a lot (most?) of the early beta FSD testers on YouTube curated their content to paint FSD in a much better light than its current state justified.
Yet there is also something to be said with FSD beta working better in some areas than others, too. I've experienced this myself by driving around Las Vegas quite a bit, where it did much better than it did in my local area (southern Utah).

Tesla still has a bunch of just routine behaviors and perception issues it needs to fix. Most concerning is that I suspect it will need a new camera suite to work properly.

I got the impression that Elon is softly/slowly modifying the dialogue in this direction from what he said in the AI Day presentation. He said that HW 3 (and the associated sensor suite) might only be, say, 200% better at driving than a human, whereas HW 4 might reach 1,000% better. So he's basically saying that you will get the intended FSD functionality with HW 3, but since it's better than a human driving, it meets the bar. IOW, HW 3 isn't going to be getting any free upgrades.

Of course, no one knows whether or not HW 3 is going to pull this off or not. My opinion is that they won't be able to; they are already "borrowing" a lot of compute from the second FSD computer in HW 3, so redundancy is not going to be an option unless they're able to highly optimize the code once it's complete. Not to mention the lack of cameras in certain key vehicle locations.

It's interesting to note how they've softened up the language of what FSD will be able to do over the years on the Tesla Vehicle Configurator page; they are no longer making the promises with purchasing the Full FSD package now that they were a few years ago.

Is this the result of Elon realizing: 1. Just that it's much harder than he thought, or 2. They know they're not going to be able to pull off the functionality they were previously promising in this hardware version?

I guess time will tell.

 

[jebinc]

I guess time will tell.

I guess it will!

 

[Vines]

What experience have folks had with FSD and non-human animals?

I was just on a drive and came to a stop on the leg of a T intersection. There was a dog in the middle of the intersection very slowly taking its time catching up to its owner on a nearby sidewalk. FSD visualized the human, but not the dog. I was concerned about the car hitting the dog, but it trotted away just as the car started to go, so I didn't have to intervene. I'm unsure what would have happened if the dog stayed where it was, but I wasn't going to put its life at risk to find out.

Curiously, closer to home on a straight road, I passed another dog walker on the parallel sidewalk. I'm not sure since I didn't glance at the visualization until the last second, but there may have been a little blob beside the visualized human. Was that the dog? If so, why didn't the previous dog show up?

I've also wondered about this out in the country more (which in our little city means only a mile or two) where we might encounter people on horseback. Further out one might even encounter cattle on the road. How will FSD handle these creatures? Does anyone have any experience with this?

I have seen my car render both dogs and turkeys. I have also had it not respond quick enough to a deer in the road at high speed, and I took over successfully.

 

[Phlier]

I have seen my car render both dogs and turkeys.

Wait... did it actually render it as an actual turkey??

Asking for a friend...

 

[mxnym]

Drove 150 miles yesterday on the highway and 10.4 performed pretty well. During rush hour traffic the ride was smoother then earlier versions. Used chill setting just to see how it worked and it worked pretty well. Fewer lane changes and acceleration on a lane change was pokey but that was expected. All in all a positive drive. Per usual the following still need to be addressed and have been widely known since forever.

1. Don't move into a faster lane when approaching an exit
2. Don't move into a faster lane it that causes the oncoming car to slow down. This drives me crazy and I sometimes cancel the lane change. Chill was better since it did fewer lane changes but chill still wants to move lanes when cars are coming up too fast. Will try more aggressive settings next time.
3. In heavy traffic just move over to the right lane when approaching an exit. Don't wait forcing me to take over or miss the exit.
4. Merging traffic entering the highway. Sometimes it cannot make up its mind whether to slow down or speed up. Usually caused at the last minute by the merging driver deciding to accelerate or slow down at the last minute.
5. FSD integration has definitely improved exit ramps. Not as many "hot" exits requiring manually slowing or disengaging. Easy to tell when my wife grabs the door handle as we approach a tight exit ramp at 60mph and the exit ramp speed sign is 25mph.

I don't have much experience with (2) because I just let it go the speed limit, but in one instance when mine wanted to change into the left lane, I saw an oncoming (not sure this is the right word for a car behind you) car and felt there was enough time as long as acceleration was brisk, so I started to press the accelerator, then FSD aborted the signal and swerved back to the middle of the lane well before the other vehicle would have been an immediate threat. I expected it to get in the left lane with my acceleration assistance and not obstruct anyone, which means either our experiences are very different or you aren't providing accelerator assistance at what may be an opportune time to do so (to be fair, I ended up really close to the vehicle I was going to be passing since I had provided accelerator assistance, so extra caution would certainly be merited in such attempts).

It's so weird that (3) is a problem in heavy traffic and tight cities (I concur that it is), because out in the middle of nowhere, when FSD needs to make a left turn, it switches lanes over a mile before the turn.

Wait... did it actually render it as an actual turkey??

Asking for a friend...

Of course it did! That's the fire holiday update, the turkey just had to cook for 11 months.

 

[Cosmacelf]

That is exactly what I thought when I went out on my first FSD drive.

I'm convinced that a lot (most?) of the early beta FSD testers on YouTube curated their content to paint FSD in a much better light than its current state justified.
Yet there is also something to be said with FSD beta working better in some areas than others, too. I've experienced this myself by driving around Las Vegas quite a bit, where it did much better than it did in my local area (southern Utah).

Good points.

I got the impression that Elon is softly/slowly modifying the dialogue in this direction from what he said in the AI Day presentation. He said that HW 3 (and the associated sensor suite) might only be, say, 200% better at driving than a human, whereas HW 4 might reach 1,000% better. So he's basically saying that you will get the intended FSD functionality with HW 3, but since it's better than a human driving, it meets the bar. IOW, HW 3 isn't going to be getting any free upgrades.

The annoying thing to me is that it wouldn't have taken a rocket scientist to figure out that the current camera suite was inadequate. Anyone who pays attention to driving and thinking like a computer would have realized that you needed better 90 degree side view cameras. Several of them.

The car is only as good as its sensors allow it to be. To take a simple example, auto park. It works, but always parks at least 6 inches away from the curb. A human can do better. Why? Because the car vision resolution at curb level sucks. 960 x 480 pixel resolution by 6 bits of color is horrible. The hardware can't support higher resolutions when driving at 80 MPH, but it sure can use it when driving at 1 MPH. Tesla really didn't think through its camera suite properly.

Of course, no one knows whether or not HW 3 is going to pull this off or not. My opinion is that they won't be able to; they are already "borrowing" a lot of compute from the second FSD computer in HW 3, so redundancy is not going to be an option unless they're able to highly optimize the code once it's complete. Not to mention the lack of cameras in certain key vehicle locations.

When I wrote my AI day summary I noted that there was a lot of optimization that Tesla could do that they just haven't had the time to do yet. I wouldn't be surprised by a 2x or more speedup once they give themselves the time to optimize a few things. That's just another reason why I call this alpha level software.

It's interesting to note how they've softened up the language of what FSD will be able to do over the years on the Tesla Vehicle Configurator page; they are no longer making the promises with purchasing the Full FSD package now that they were a few years ago.

Is this the result of Elon realizing: 1. Just that it's much harder than he thought, or 2. They know they're not going to be able to pull off the functionality they were previously promising in this hardware version?

So, I've had a refreshed Model X on order since February but when my turn comes up, I'm now going to defer until I see a new sensor suite. The fact that non-radar AP folks are less than impressed with freeway autopilot also freaks me out (I have radar in my 2019 Model X). Freeway AP works flawlessly for me, and it sounds like they've gone backwards on that.

 

[TIppy]

You can set it as an option. In the morning when I leave, FSD picks "work" as the destination by default.

Doesn't that actually set a route, which FSD would follow if enabled? That's a little different than what I took him to say. I thought he said if no route was set when you enabled FSD it would drive to a location it chose. If you enable FSD without a route, it drives but no route is displayed.

 

[FSDtester#1]

Yesterday I decided to just use AP, instead of FSD. Mainly because I got the booster on Wednesday and had mild headache. A few things I learnt.

- AP works better than production AP. Lane keeping is better and doesn't get confused when the lane splits.
- AP worked like before in rain. None of that "severe weather detected" stuff. This is really a great discovery for me since I need AP to drive at night when raining. AP definitely sees the lines and can make out lanes better than me. So, from now on - until FSD improves in these conditions - I'm just going to switch to the AP profile and use AP when it is raining (esp. at night)
- The blue lines it shows are rock solid in rain at night and I've no idea how it can figure out the lines that well. With FSD the lines were all over the place.

How do we switch to just ap?

 

[Mardak]

car vision resolution at curb level sucks. 960 x 480 pixel resolution by 6 bits of color is horrible

Those numbers would indeed only have a per-frame raw size of 0.3MB. Do you think something 6x more data at 1.8MB/frame would be better? Say 1280x960 12bit HDR?

 

[TIppy]

How do we switch to just ap?

Turn off the FSD Beta toggle in autopilot setting.

 

[Vines]

Wait... did it actually render it as an actual turkey??

Asking for a friend...

Hard to really say until I get a camera installed in the car. I believe they were rendered a a 4 legged dog like object.

FSD successfully navigated through the flock, even when they started running in different directions as the car approached them.

 

[EVNow]

Is this the result of Elon realizing: 1. Just that it's much harder than he thought, or 2. They know they're not going to be able to pull off the functionality they were previously promising in this hardware version?

(1) Yes.

But more importantly - they want to limit what they are promising so that at some point they can book the revenue. No CFO / Master of Coin wants to have this obligation dangling indefinitely.

ps : On another note - figured out it was probably better to just give the accelerator a push to navigate the roundabouts than disengaging/reengaging at every roundabout. I tried at 3 or 4 roundabouts today. Will have to try on a few more.