FSD v12.x (end to end AI)

[zoomer0056]

Guess AI condones a little drinking and driving.

View attachment 1026143

Wait ... his hand is on the wheel ... what's up with that. Still, some Darwin Award recipients may look at this pic and put others in danger.

 

[Jeff N]

This squeezing in a narrow shoulder past vehicles for a right turn reminded me of 11.x behavior of folding mirrors in narrow situations. Have people noticed if 12.x still has that ability?

I have driven through multiple narrow road situations where V11 would have folded the mirrors and V12 has not done that yet.

That’s one of the most frustrating things about FSD. I remember my wife asking once why it kept making the same mistake and I explained to her that it had no memory and therefore no ability to learn.

Having a memory and learning would imply the ability to modify the NN on the vehicle which I don’t see happening in the near future.

I’m not sure that’s true. The FSD NN seems to be fed navigation path data that weights/guides it on when to turn and perhaps even includes incomplete lane metadata guidance. If the NN could detect cases where it guessed wrong at lane metadata and then had self-correct it could, maybe?, output those corrections. The corrections could then be saved outside the NN and used to enhance navigation path input back into the NN during future drives by the same car. The car could also feed those corrections back to Tesla and if enough cars drive through the same path and provide the same correction then perhaps the data could be permanently integrated into the navigation data periodically downloaded to all cars.

Or not. I don’t understand the FSD architecture well enough to know if that makes any sense.

 

[FSDtester#1]

Thanks for the update. I don't think I remember Tesla ever releasing a version to the public on a weekend though. As hard as Elon is said to work his employees, it seems like autopilot/FSD team gets the weekend off.

They do like to release on Fridays though which I find odd that if it turns out to be a terrible version that they would let it play out over the weekend before they pull it. (If it was truly bad, I'm sure someone would get to it even on a weekend but still)

Most of the updates seem to come on a late Saturday, or usually late Sunday night.
I have been late to work Monday a few times, waiting for the update to finish.

 

[Yelobird]

Most of the updates seem to come on a late Saturday, or usually late Sunday night.
I have been late to work Monday a few times, waiting for the update to finish.

Beats the dog ate my report excuse. I’m using that!

 

[lzolman]

A shame we can’t do that to the human brain. My doofus weighting is too high; would like to crank that back a bit.

We can. It's known as meditation.

 

[aronth5]

The more I use V12, the more I have that feeling that it's the real deal: the final boss of FSD

Some of the maneuvers and decisions V12 makes create lasting memories

V12 is getting scary good at fixing its wrong lane selections

The missing piece is some form of lane or area memory that helps it avoid the same mistakes every time

But, it's better to first be good at fixing mistakes

Thanks for all your feedback.
Assuming V12.3 continues the improvement how far away do you believe FSD is from being safe enough to take elderly parents to places like the doctors office or the grocery store? Assuming the elderly person can handle parking at the end of the ride and/or a graceful FSD handoff to the driver in the event of an edge case. For instance an accident scene with police offices directing traffic. The objective being to allow the elderly to keep their cars a bit longer which can be very important to them.

 

[zoomer0056]

how far away do you believe FSD is from being safe enough to take elderly parents to places like the doctors office or the grocery store?

You can take them now. Just be sure they're strapped in, and warn them you might have to disengage a couple of times.

 

[Supcom]

Thanks for all your feedback.
Assuming V12.3 continues the improvement how far away do you believe FSD is from being safe enough to take elderly parents to places like the doctors office or the grocery store? Assuming the elderly person can handle parking at the end of the ride and/or a graceful FSD handoff to the driver in the event of an edge case. For instance an accident scene with police offices directing traffic. The objective being to allow the elderly to keep their cars a bit longer which can be very important to them.

Your use case, which is to allow someone to continue driving when the person would otherwise need to give up their car, requires an L4 system with an ODD that will includes their driving requirements (geofencing, time of day, weather, etc). An L2 or L3 system requires a competent driver ready to take over at any time.

Edge cases tend to be the more difficult scenarios and may require quick reaction to know when to initiate a takeover. If the driver is able to handle those, then the driver should be able to use an L2 system. But, I think that you are describing a driver who actually cannot do so.

If the parents are approaching this situation, it might be best to get them comfortable using rideshare services, like uber, so that they can transition from driving to riding while keeping their independence.

 

[rlsd]

Now that's some validation (advertising?)...

https://twitter.com/x/status/1766206509191438825

Home price in Chuck Cook's neighborhood will increase very fast. Buy now.

 

[JulienW]

Home price in Chuck Cook's neighborhood will increase very fast. Buy now.

Kinda the same way our used Tesla's have become an appreciating asset over the last year just like Elon said.

 

[mgs333]

Most of the updates seem to come on a late Saturday, or usually late Sunday night.
I have been late to work Monday a few times, waiting for the update to finish.

Are you referring to when you get the updates? I was thinking back to when the releases first drop.

 

[rlsd]

So a follow up question would be does anyone know the structure of the V12 NN? How many hierarchy layers and how many weights total?

My next weekend project: Open my car frunk to take the computer CPU out and cut it into pieces to see how the structure looks like.

 

[JulienW]

My next weekend project: Open my car frunk to take the computer CPU out and cut it into pieces to see how the structure looks like.

Isn't the FSD computer in the cabin?

 

[spacecoin]

My next weekend project: Open my car frunk to take the computer CPU out and cut it into pieces to see how the structure looks like.

The best FSD-computer is no FSD-computer?

 

[spacecoin]

I bring up chess because it's simpler in the hopes that people can better understand how reinforcement learning for predicting end-to-end controls progresses. The core part is that the same network size can be improved with examples of where it performed poorly as well as examples of how it could do better. Chess neural networks can beat humans even when only considering the current board position to predict which one action to take, i.e., it doesn't need to brute-force explore any next possible moves. (Yes, it'll play even better if it can think about future board positions too, but that doesn't seem as applicable to 12.x.)

FSD Beta 12.x has more complications than chess such as real-time processing of the video at 36fps, but for each frame, it's still predicting the appropriate control action, so for a relatively "early" network like 12.2.1, this could result in decision wobble if each of the 36 frames don't predict the same best control. Similarly, human disengagement and correct driving examples provide reinforcement learning data for the end-to-end network to improve controls for the next release without requiring a larger neural network that might approach hardware compute limits. Instead of brute-forcing with a chess simulator to find better behaviors, Tesla can deploy 12.x to find real-world examples of where humans drive better.

If this is how one thinks one develops safety critical systems, think again. Your reasoning above reminds me of James Douma's "expert" insights.

 

[Mardak]

The FSD NN seems to be fed navigation path data that weights/guides it on when to turn and perhaps even includes incomplete lane metadata guidance

Yeah, given that it seems like Tesla has improved mapping by including remotely provided navigation data that seems to include lane counts and connectivity (e.g., 3 lanes in your direction with the 2 right lanes continue to another road). This additional data seems to replace any (incorrect or missing) lane data that was part of the map update (e.g., NA-2023.44-14828). Both previous 11.x and current 12.x seem like they take the navigation data as inputs, and the various FSD Beta can do better with correct map data but also get confused by wrong data, and ideally end-to-end figures out when to trust map data based on what's actually visible.

So there is the possibility that each vehicle driving through an area detects a mismatch with the current map data, e.g., there's actually 4 lanes not 3, and this could be sent back to Tesla servers or kept on the vehicle. Tesla servers could detect that multiple vehicles passing through are consistently reporting a different value to then send back to individual vehicles when navigating and eventually as part of map updates. If Tesla does make use of locally detected map data from previous trips, the tricky part is now there's 3+ sources of data (map, navigation, previous trips), and all of them could be wrong, and merging them is not always straightforward.

Overall, this does seem like a nice way for Tesla to continuously improve 12.x behaviors even between software updates that have improved neural networks.

 

[powertoold]

Assuming V12.3 continues the improvement how far away do you believe FSD is from being safe enough to take elderly parents to places like the doctors office or the grocery store? Assuming the elderly person can handle parking at the end of the ride and/or a graceful FSD handoff to the driver in the event of an edge case. For instance an accident scene with police offices directing traffic. The objective being to allow the elderly to keep their cars a bit longer which can be very important to them.

I believe V12 is the most promising approach for generalized fsd, and it's already performing very well. You will see what I mean when you use V12.

But as we know from prior versions, any small issue can become a fatal flaw over time

As for police directing traffic and whatnot, I think it's a tractable problem with V12. I think lane selection in heavy / obstructing traffic is a harder problem than following hand-arm signals, but I could be wrong.

At the end of the day, the real answer is something like: yes, I think V12 will eventually be the approach that leads to 3-10x human safety performance (along with all the smoothness and naturalness that V12 brings) -- no, it won't be perfect and understand every little thing, but it will be safer than humans by far, and the real question will then be: will society accept a system that can fail but is 3-10x safer than humans?

As for the timeline, it's unclear, but again, if you use 12.2.1, it seems very close (2-3 years for this generalized robotaxi, perhaps the first HW5 cars?)

 

[powertoold]

BTW BTW!!

I think there are a variety of reports and testimonials for V12 because people are using all sorts of settings

BUT

I've found that Average auto-max leads to the fewest interventions and disengagements - overall fewer interventions and disengagements than 11.4.9!!

Average auto-max is so great; it's a little slow at times when there's light traffic and no lead car, but it will drive with the flow when there are more cars

Stop sign behavior is great as well! Still not perfect but much better than 11.4.9 IMO!

I'm loving 12.2.1, especially after the camera calibration! I know it's probably placebo or using the Average auto-max. It's got me excited, and my wife doesn't say or complain at all about any of the driving! Although you can totally tell the freeway stack feels unnatural compared to city-streets.

 

[sleepydoc]

Wait ... his hand is on the wheel ... what's up with that. Still, some Darwin Award recipients may look at this pic and put others in danger.

They would put people in danger without the pic

 

[sleepydoc]

Isn't the FSD computer in the cabin?

no, @kabin has it