FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Dewg]

Just got home this morning and engaged autopark into my garage and it wouldn't work. I heard it engage Drive and the immediately engage Park. I tried several times, still wouldn't pull forward. I parked manually and then attempted summon back out, and that worked. Tried forward again, nope.

 

[ParaCanary]

Same problem. I guess they updated the software. I solved it by changing the - not hold continuously the summons button - on the tesla screen. Also do not stand too close the the uss. It now goes in and out of my garage without a problem - much better than before.

 

[pimfram]

My true test for FSD will be if my mother screams on the trip to our Christmas host today.

The car attempted to take exit lanes twice as they diverted on the interstate. It was much better on the surface streets after exiting the highways. My mother was not impressed. It was raining so there was no FSD usage on the return trip.

 

[ArchHamster]

OK... I officially don't get it. Why do cars WITH USS NOT get high-fidelity park assist since it is currently only camera based?

The white night response:

HFPA hasn't proven itself more reliable to uss yet, displaying both uss and HFPA may lead to significant user confusion on which one to trust, and they may still need pure uss to help train HFPA and not confound the training data.

Dark hamster response:
If uss was enabled along with HFPA, then there would be a billion comparison videos which may not paint ussless cars in a positive light. Uss would provide a way, at a glance, to critique HFPA.

Christmas ham ham response:

Tesla feels bad for how much they thoroughly f'ed ussless cars in the past year or so, so they are giving them something nice for a change.

 

[Dewg]

Another possibility - HFPA is based on HW4 and trained on the higher res cameras. Does anyone have HFPA on HW3?

 

[Twiglett]

Another possibility - HFPA is based on HW4 and trained on the higher res cameras. Does anyone have HFPA on HW3?

or alternatively- who cares and just uses their mirrors like everyone else
seems like every time I rely on the cameras I end up with busted rims
There’s a lesson in there I’m sure

 

[APotatoGod]

Another possibility - HFPA is based on HW4 and trained on the higher res cameras. Does anyone have HFPA on HW3?

The new 3D view when parking is definitely present on HW3 Model 3s without USS.
Though I suppose there's a chance (but unlikely) that it's an MCU2 vs MCU3 difference?

 

[hybridbear]

or alternatively- who cares and just uses their mirrors like everyone else
seems like every time I rely on the cameras I end up with busted rims
There’s a lesson in there I’m sure

I've found the best for our S is a combination of the mirrors moving down to positions we've saved along with the cameras. I like the fender cameras to confirm spacing to the garage pillar we park next to at our apartment building. But the mirrors are the best for avoiding curbs when parallel parking or parking at the edge of a parking lot. And the USS are unbeatable for parallel parking & getting close without hitting any other vehicles or obstacles.

 

[kabin]

I had a chance to test the stationary photo radar again and still no joy. It's been at the same intersection for at least 10 years. No indication on the map or in the navigation dialog box. Things that make you go hmmm.

 

[Supcom]

Another possibility - HFPA is based on HW4 and trained on the higher res cameras. Does anyone have HFPA on HW3?

I have HFPA on a HW3 USS-less MY.

 

[AndrewZ]

11.4.9 highway performance is impressive, other than overly aggressive stopping / accelerating

I'd say 11.4.9 can go 100k miles on some highways without a 50%+ at fault accident (minus complicated construction)

The most recent Vehicle Safety Report, at the end of 2022, had a rate of one accident every like 5million miles for vehicles on Autopilot

 

[EVNow]

The video looks like it was taken by a rear seat passenger, was it originally posted by wholemars or someone else?

Wholemars iirc - one of the YT guys from Seattle visited him and they went on a drive and filmed it.

 

[powertoold]

The most recent Vehicle Safety Report, at the end of 2022, had a rate of one accident every like 5million miles for vehicles on Autopilot

I'm talking unsupervised

 

[sleepydoc]

The white night response:

HFPA hasn't proven itself more reliable to uss yet, displaying both uss and HFPA may lead to significant user confusion on which one to trust, and they may still need pure uss to help train HFPA and not confound the training data.

Dark hamster response:
If uss was enabled along with HFPA, then there would be a billion comparison videos which may not paint ussless cars in a positive light. Uss would provide a way, at a glance, to critique HFPA.

Christmas ham ham response:

Tesla feels bad for how much they thoroughly f'ed ussless cars in the past year or so, so they are giving them something nice for a change.

I have a suspicion that they still consider USS more accurate for distances and they also were focused on getting V1 working on it’s own and haven’t put any work into combining it with USS as that integration would not be trivial and likely take a reasonable amount of work.

 

[FSDtester#1]

Turns out Santa was using FSDb last night for deliveries,
and due to poor map data, errant turns, slowdowns, no wiper functionality, and near aerial collisions, many kids woke up to no presents and their parents noticed some scratches and dents on their vehicles and homes. Found the article in the LA Times, so take it at face value...


Norad is apparently leading the investigation

 

[aronth5]

FSD, park assist and rear view camera don't work today. Reset display, cycled power twice. No luck. Any ideas? On my old M3 my camera didn't work once but FSD still did. I wonder if that prerequisite has changed?

 

[Dewg]

FSD, park assist and rear view camera don't work today. Reset display, cycled power twice. No luck. Any ideas? On my old M3 my camera didn't work once but FSD still did. I wonder if that prerequisite has changed?

Can you go into service mode and check the hardware? Do all the systems show green? Check all the camera feeds while in there.

If everything is green, try reinstalling the software.

Then try pulling your USB drive and test.

 

[hybridbear]

If everything is green, try reinstalling the software.

Do you still have to be outside a Service Center connected to their wifi to redeploy the software? We had to do this once back in Jan 2022. It wasn't as bad as it could have been for us because the nearest Service Center is only 15 mins away. But it still took over 60 mins round trip to redeploy the software update that had issues. This post explains how to redeploy the firmware.

Then try pulling your USB drive and test.

Right now we get frequent "autosteer aborted for system error" messages while using FSD Beta. Often the option to use TACC or FSD Beta will not return until the USB for Sentry Mode/TeslaCam is removed for some time. Sometimes a two scroll wheel reboot is required. So there is something about pulling the USB that affects the car.

 

[Scott Chapman]

So I had my first couple of "incidents" with 11.4.9 driving home last night

1. I got a hard brake on the highway - I was in the passing lane, and just approaching a truck on the right to pas it. It seemed to get confused and hit the brakes HARD. I saw it put up a stop line that appeared, disappeared, reappeared. I was able to hit the accelerator to "fix it" but it was very alarming to say the least.
2. it had a difficult time figuring out when to merge into the off ramp lane a couple of times. Basically it did a hard adjustment either early or late. Wasn't smooth.

Anyway, it was at night, and there was a lot of fog. I suspect that was related.

 

[hybridbear]

So I had my first couple of "incidents" with 11.4.9 driving home last night

1. I got a hard brake on the highway - I was in the passing lane, and just approaching a truck on the right to pas it. It seemed to get confused and hit the brakes HARD. I saw it put up a stop line that appeared, disappeared, reappeared. I was able to hit the accelerator to "fix it" but it was very alarming to say the least.

Was the truck infringing on your lane or moving closer to the line diving you lane from its lane at the time of the hard brake? I've experienced that FSD Beta can be alongside a semi truck/trailer & if the semi angles toward its lane FSD Beta will slam on the brakes even though the truck is still in its own lane.