Any other FSD improvements with 2020.48.26 / holiday update?

[Mardak]

Looks like traffic light detection for US non-FSD-beta people improved with 2020.48.26 no longer relying exclusively on the main camera. Here's an intersection where Autopilot would definitely fail to pick up the red light if stopped first at the stop line and would also miss the green if stopped slightly past the stop line:

With the latest software, the visualization shows these 2 green lights easily suggesting it now uses the fisheye camera. I suppose potentially it could still be using the main camera and detecting the green glare, but there was also a light visualized directly to my right that seemed to disappear when it was out of view of fisheye -- reinforcing that it's the wide camera being used and not the pillar cameras (which seems to be used in Europe).

Have people also noticed other unlisted FSD improvements with the holiday update?

 

[HighZ]

Have people also noticed other unlisted FSD improvements with the holiday update?

I feel that phantom braking is reduced (but not completely eliminated) and it seems to be slowing for stops (yellows, reds and stop signs) more naturally and gracefully.

 

[Mardak]

it seems to be slowing for stops (yellows, reds and stop signs) more naturally and gracefully.

At least for the stop line detection, maybe this is also due to using wide fisheye camera for more confident predictions of where to stop? Here's a screenshot from green's video:

Compared to main, the line isn't visible that much longer in fisheye, but the wider view does see more of the line which could be useful if another vehicle is blocking the view directly in front. This is important for smooth stopping because otherwise it might need to guess based on a line position predicted from a few moments ago.

If the fisheye camera is indeed being used, I wonder if there's a whole set of FSD improvements from that. I'll keep an eye out for things that benefit from seeing wider.

 

[Mardak]

Here's an example of traffic light detection working fine for a wide intersection:

When pulled up to the line, the left side traffic light and even the street lamp overhead aren't in view of the main camera, but the car renders the left light color fine. So Autopilot is at least visualizing colors of traffic lights that are up higher and/or more to the side of the main camera than before. People can probably most easily confirm by noticing when a green light changes to a blank light in the visualization as you drive under.

At least testing so far, there's no wide-fisheye-related driving behavior changes, e.g., predicted path or lane lines, so in some sense, these visualization changes are a slightly improved "FSD sneak preview" in this year's holiday update.

 

[Snerruc]

I find the new release very useful. No phantom braking, overall Autopilot rock steady and stoplights much better. Nagging has almost disappeared after being worse the last few updates.

 

[Mardak]

Stop sign detection seems to now be using the wide camera like it does now for traffic signals:

This stop sign isn't visible from the main camera, but it's directly to the right of the "STOP" road marking just past the sidewalk. I made sure to avoid the memory aspect of Autopilot by reversing a little bit at a time until the stop sign appeared in the visualization when in drive.

However, speed limit sign detection seems to still be reliant on the main camera:

Same behavior as above reversing slowly until the visualization showed the sign, and this case it only appeared at this point when in view of the main camera. Kinda neat to see Autopilot able to read the speed without the whole sign in view.

I wonder how Tesla is deciding which camera to use for detecting traffic lights and stop signs as using all 3 front cameras could easily result in duplicates in the visualization. One of the reasons for the birds-eye-view network used in the FSD rewrite is to take these individual camera predictions to decide on a unified view. So perhaps for non-FSD-beta, it's just doing some basic filtering, e.g., if the predicted distance to the signal is within some distance, only use the output from the fisheye neural network.

 

[blkswn]

I had an instance on AP where it slowed for a large dip on a main road. Not sure if it was a one off occurrence or if it's trialing things picked up from FSD beta users. Didn't slow for a smaller/narrow dip in a neighborhood street. Anybody else have this happen?

 

[Mardak]

I tested turning at intersections with traffic lights to see if they correctly show colors when out of view of the fisheye camera, and I'm pretty confident the wider visualization is from the wide front camera and not the pillar cameras. Here's what it looks like for FSD beta from James Locke:

Notice how there's two green left turn lights visualized in FSD beta even though the car is no longer even facing the traffic lights.

So did Tesla add fisheye neural network light detection because it was good enough for general release? Or is it more like the original traffic light support to collect fleet data with drivers confirming to train birds-eye-view networks used for FSD?

 

[Mardak]

I had an instance on AP where it slowed for a large dip on a main road

Are you talking about something like these to channel water? Here's two along both sides of Chapman when crossing Western:

 

[blkswn]

Are you talking about something like these to channel water? Here's two along both sides of Chapman when crossing Western:

View attachment 622304

Yes, exactly like that except it wasn't at a large intersection, but later on down the road.

Also, I just remembered another event where the car slowed to 22 mph for slightly raised railroad tracks going across the street once before as well while I was on AP. I thought it was phantom braking at first but then noticed the raised tracks which look like a wide speed bump. It didn't fully brake like a typical phantom braking scenario, but rather just slowed a bit as if through regen from 35 mph to 22 mph which I remember thinking was odd as I expected it would've slowed to 15 mph as seen on the FSD beta videos. Both of these events happened on the latest 2020.48.26 update.

 

[Mardak]

@verygreen Are you able to enumerate and render the godot-based visualization assets that are now used in general production builds? FSD Beta has had the new visualizations for a while, but now that I've gotten the holiday update with the super detailed car avatars, I've noticed some differences:

The old visualization had a collar/base at the bottom of the stop sign pole with a slightly darker shade of red, and the traffic lights used to be full cylinders for each light instead of the sliced cylindrical segments. The new godot visualizations also no longer support 3-finger dragging to reposition the "center" of the visualization and only supports zooming + rotating, but unclear if this was intentionally removed or overlooked during a rewrite.

Separately, do you get any insights into what data is fed into the visualizations as my testing earlier in this thread seems to indicate traffic lights and stop signs are from fisheye neural network outputs while speed signs are still main camera, and neither use pillar cameras in the US.

 

[thewishmaster]

I also noticed additional traffic light visualizations where even I couldn’t figure out which light it was visualizing from where I was sitting. Also seems to do a slightly better job accelerating to freeway speeds on on-ramps, e.g. no longer tries to enter the freeway at 55-60mph for me (small sample size of only a couple drives though).

 

[Mardak]

I also noticed additional traffic light visualizations where even I couldn’t figure out which light it was visualizing from where I was sitting.

Haha yeah, I've found some new traffic lights from the visualizations after thinking Autopilot had a false positive. Turns out it spotted a dimly lit traffic light that was about half the size of a regular traffic light far on the opposite side of the street, and I would assume this light is for the crosswalk instead of the standard pedestrian crossing light.

I have encountered several actual false positives with the fisheye light detection especially at night for green-ish lights, e.g., holiday green lamps and blue emergency call boxes off the side of the road. So it does seem more likely that Tesla turned on the wider detection to collect training data to improve FSD neural networks.

 

[thewishmaster]

I have encountered several actual false positives with the fisheye light detection especially at night for green-ish lights, e.g., holiday green lamps and blue emergency call boxes off the side of the road. So it does seem more likely that Tesla turned on the wider detection to collect training data to improve FSD neural networks.

You haven’t lived until your car tries to stop at a green light that it saw about 200-300 feet off the (opposite) side of the freeway you are on ... going to try to replicate the next time I take the same route. It was aimed just right to confuse the car

 

[Mardak]

Not sure if this qualifies as an improvement, but I believe forward arrow traffic lights are now identified differently from plain circle lights and left/right arrows. Most of the time, the forward arrows are dimmer indicating Autopilot doesn't think it's for your current lane. So again maybe this is just part of the data collection now that the neural network is identifying the forward arrow and needs the human to confirm it's for you.

 

[S4WRXTTCS]

I haven't tested stop light control, but I have driven with TACC/AP/NoA pretty extensively since the holiday update.

TACC seems a less prone to false braking. Only a few minor slow downs, and nothing dramatic.

AP seemed pretty solid, and auto lane changes seem pretty solid as well.

NoA remain just as horrible broken as it always has been. I haven absolutely no idea how anyone is getting good results with it. There is still way too much delay in changing lanes. Then to make matters worse there is a section on I5 north of Seattle where NoA thinks it can't be in the passing lane. It constantly wants to get over for navigation reason. So I'm stuck doing 60mph in the middle lane because if I get in the passing lane to pass the NoA will throw me back over. The cancel button doesn't work.

I just got done embarrassing myself because I stubbornly kept trying to use it. When I turned it on it immediately turned on the blinker to get over despite the fact that I was just about to pass a small truck in the middle lane. So here I am passing the truck with my right blinker on the entire time. I should have given up on it, but I was trying to figure out what it was doing.

 

[LowlyOilBurner]

I haven't tested stop light control, but I have driven with TACC/AP/NoA pretty extensively since the holiday update.

TACC seems a less prone to false braking. Only a few minor slow downs, and nothing dramatic.

AP seemed pretty solid, and auto lane changes seem pretty solid as well.

NoA remain just as horrible broken as it always has been. I haven absolutely no idea how anyone is getting good results with it. There is still way too much delay in changing lanes. Then to make matters worse there is a section on I5 north of Seattle where NoA thinks it can't be in the passing lane. It constantly wants to get over for navigation reason. So I'm stuck doing 60mph in the middle lane because if I get in the passing lane to pass the NoA will throw me back over. The cancel button doesn't work.

I just got done embarrassing myself because I stubbornly kept trying to use it. When I turned it on it immediately turned on the blinker to get over despite the fact that I was just about to pass a small truck in the middle lane. So here I am passing the truck with my right blinker on the entire time. I should have given up on it, but I was trying to figure out what it was doing.

Yep, it needs to be faster to change lanes, and merge quicker. For the most part, it seems to wait till the last minute to move into traffic from an on ramp, even when there is no traffic in the lane it’s trying to merge into. Those are my major annoyances.