FSD v12.x (end to end AI)

[enemji]

Basic NN question, sorry to hijack: how does a neural net actually work? So let's say it's trained on billions of videos so it "knows" what to do in most situations. What does that "knowledge" actually look like?

I mean, the NN can't just sit there and continuously look at all the videos all the time and choose something to do, that's impossible. It must have some kind of generalized understanding that it actually uses in different contexts. Can anyone probe the NN and really see what's actually going on, in the immediate functioning of the system? Can we look at its "understanding" (whatever form that really is) and see a finished form of the net result of all its training?

I guess another way of asking is -where- does it store -what- information, and how is that information processed from perception to action?

I know, go take a class in computer science, or at least google this, but y'all seem really knowledgeable about how it all works, so maybe someone can provide an executive summary for newbies?

My biggest problem is understanding how billions of frames of sequential images made up of pixels (video) can make an impression on some mysterious NN "program" that can be separated from the training and loaded into a pretty simple computer in the car.

It is algorithmic, not heuristical and not searching through a database attempting to determine the situation and the next step.

 

[Mardak]

It shows that we are getting to the point where AI can process very large amounts of data, with very high recall

Figure 8 shows a comparison of Gemini Pro 1.5 and GPT-4 Vision for recalling information from video where they say Gemini supports up to 180 minutes while GPT4 is only 3 minutes. Either number seem pretty excessive for driving until noticing the video clips are sampled at 1fps, so if instead using Tesla's cameras at 36fps, GPT4 is 5 seconds of one camera whereas Gemini could do 37.5 seconds of 8 cameras.

Indeed overall it's also impressive that Gemini "1.5" Pro is comparable to Gemini "1.0" Ultra where presumably Ultra is much larger yet neural network architecture improvements like Mixture-of-Experts allows for better quality without increasing inference compute requirements significantly. This could potentially lead to say end-to-end "1.5" on HW3 (or even HW2.5?) performing as well as HW4 end-to-end "1.0" architecture.

 

[drtimhill]

I get your point about design versus deployed. But the vehicle is still L4 during testing, before certification. It is not quasi-L2 or L2 during testing with human supervision. It is supervised L4. SAE levels are very clear that L4 is not L2 when there is a safety driver. In fact, J3016 specifically says that it does not matter if the vehicle is under testing or deployed commercially, the level is whatever the design intent is.

Here is what J3016 says on this:

"The level of a driving automation system feature corresponds to the feature’s production design intent. This applies regardless of whether the vehicle on which it is equipped is a production vehicle already deployed in commerce, or a test vehicle that has yet to be deployed. As such, it is incorrect to classify a Level 4 design-intended ADS feature equipped on a test vehicle as Level 2 simply because on-road testing requires a test driver to supervise the feature while engaged, and to intervene if necessary to maintain operation." (p. 36)

Interestingly, I went back and looked at my (very old, early release) copy of this document, and could not find this qualification (it's in the new exactly as you quote it). My guess is I wasn't the only one who spotted this confusion, and this text was added to clarify the intent of the various levels. However, I do think the phrase "design intent" is vague at best (I could find no clear definition in the standard). However, I agree with your interpretation.

 

[spacecoin]

I have not heard that but I am sure elon basically said something along the lines of “it is similar to L2 or L3” but not officially categorizing it as a specification of the FSD

What about this:

Elon: "Full autonomy next year" (This was 2020).

Interviewer: Level 5? Elon: Yes.

 

[AlanSubie4Life]

What about this:

Elon: "Full autonomy next year" (This was 2020).

Interviewer: Level 5? Elon: Yes.

Shows @Daniel in SD is right. L5 design intent! They are not working on any other product secretly.

 

[spacecoin]

Shows @Daniel in SD is right. L5 design intent! They are not working on any other product secretly.

Marketing intent is not design intent... The boy who cried FSD for ten years. Fraudulent or clueless? You decide.

Related: How many tranches of his comp plan would he have met if FSD wasn't == robotaxi? Most analysts have the core auto business valued at around $75-80 per share.

 

[kabin]

Per Isaacson's book Elon tends to overstate with wild projections for investment dollars (and stock price). He doesn't seem to drop the hammer on the FSD team when they fail to meet his goal year after year after year so he knows the goal is BS. Part of the problem is his stubborn and poor system design decision making.

 

[Artful Dodger]

The boy who cried FSD for ten years. Fraudulent or clueless? You decide. Related: How many tranches of his comp plan would he have met if FSD wasn't == robotaxi?

His 2018 Comp plan also required reaching 12 of 16 miles possible milestones for TTM revenue and EBITA, which happened over just 2.5 years (the plan gave him 10).

Your misplaced attempt to spin this as 'fraud' reveals nothing about the company's progress, but much about you.

 

[spacecoin]

I call BS. Elon wouldn't have gotten past tranche 3 if the market cap was under $200B. Core auto part of market cap TODAY is about 240B.

https://ml.globenewswire.com/Resource/Download/edfcfa2e-bcb6-43ed-ab53-4b08b3bf21ca?size=0

 

[Artful Dodger]

Giving Alvie a Ride on Tesla Full Self-Driving Beta 12.1.2 | Whole Mars Catalog (12 min ago)

 

[JB47394]

Giving Alvie a Ride on Tesla Full Self-Driving Beta 12.1.2 | Whole Mars Catalog (12 min ago)

A good drive. Kinda boring because it didn't screw up anything. Unexpectedly, 12.1.2 negotiated with other drivers at intersections quite well.

These are the interesting moments that I spotted.

01:48 Maneuvering past a stopped car and stopped delivery van with oncoming traffic.
06:28 Waiting for cross traffic at a stop sign, then waiting againn for a bicycle trailing the car,
06:45 Moving right to make room for opposing traffic turning onto the Tesla's lane,
06:45 Slight hesitation on a right turn at a T intersection,
09:07 Surprisingly casual pass of a pedestrian crossing the road
10:35 Car pops out from the right, with its nose in the road
10:48 Car in front drifts forward a little while stopped at light. Tesla doesn't move up. Car drifts forward again. Tesla moves up.
12:49 Moving left across multiple lanes to park on the left curb for left destination on a one-way street

 

[kabin]

A good drive. Kinda boring because it didn't screw up anything. Unexpectedly, 12.1.2 negotiated with other drivers at intersections quite well.

These are the interesting moments that I spotted.

01:48 Maneuvering past a stopped car and stopped delivery van with oncoming traffic.
06:28 Waiting for cross traffic at a stop sign, then waiting againn for a bicycle trailing the car,
06:45 Moving right to make room for opposing traffic turning onto the Tesla's lane,
06:45 Slight hesitation on a right turn at a T intersection,
09:07 Surprisingly casual pass of a pedestrian crossing the road
10:35 Car pops out from the right, with its nose in the road
10:48 Car in front drifts forward a little while stopped at light. Tesla doesn't move up. Car drifts forward again. Tesla moves up.
12:49 Moving left across multiple lanes to park on the left curb for left destination on a one-way street

That was a pretty good drive. I wonder if some of the occasional assertiveness is a bit too cavalier now. It didn't seem to register or react to the car @10:35 popping out from the right. I might have wanted to slowed a bit to assess the vehicle's intentions, make sure I had driver eye contact, or at least readied the brakes.

 

[JB47394]

It didn't seem to register or react to the car @10:35 popping out from the right. I might have wanted to slowed a bit to assess the vehicle's intentions, make sure I had driver eye contact, or at least readied the brakes.

Well, I described it as "popping out", but if you watch carefully, the car is visible early enough to understand that it was stopping. FSD saw that, maintained 16 mph until it passed the car, then accelerated to 21 mph. I assume that if the car had not been there, it would have accelerated earlier.

I wonder if some of the occasional assertiveness is a bit too cavalier now.

I've been wondering that myself, but I think we'll only get a good sense of that when we're the ones being driven around.

 

[enemji]

What about this:

Elon: "Full autonomy next year" (This was 2020).

Interviewer: Level 5? Elon: Yes.

NOPE. He does not say L5 or Level 5. He says Full Autonomy and/or Full Self Driving. I know elon more than he knows himself

 

[Knightshade]

NOPE. He does not say L5 or Level 5. He says Full Autonomy. I know elon more than he knows himself

Interviewer: And we are discussing Level 5 autonomy?
Elon: Yes

 

[FSDtester#1]

Geez, I was out of town for the week. Just logged in to see 15 pages behind.
After a few updates to the ignore button, got it down by 5 pages.
Nonsense arguments being made, and replying to the offenders is just giving them more fuel for their straw man arguments. How about a new thread for these discussions for whoever may be interested in these OT discussions.
Does anyone have any information about a potential expansion date for V12 to posters in this thread?

 

[kriskros91]

I wonder how V12 does with inaccuracies with the Tesla map. There are areas on my normal work commute where the nav thinks the speed limit is 45 mph on a toll road that routinely people are driving 75 mph on. I have to manually scroll up every time. From a video Omar did he couldn't scroll or change the speed when he was on AUTO MAX speed settings which would be really annoying to me.

 

[JulienW]

Geez, I was out of town for the week. Just logged in to see 15 pages behind.....

Nah you're not. You should consider it 15 wasted pages saved.

 

[enemji]

Interviewer: And we are discussing Level 5 autonomy?
Elon: Yes

LOL. Elon does not say Level 5. He responds with a yes, it will be full autonomy. Show me a clip where he utters the words SAE Level5.

 

[diplomat33]

Show me a clip where he utters the words SAE Level5.

https://twitter.com/x/status/1753437704006820180