FSD v12.x (end to end AI)

[powertoold]

Chuck said it in a Youtube video. He was showing his drone camera and said, "That absolutely looks like manual driving there". This was back in December.

Yea I don't know why this is so difficult to understand. Perhaps during testing, the driver had to disengage and complete the maneuver manually? And maybe they're trained to constantly keep their hands on the wheel as if they're manually driving? Also, "looks like manual driving" doesn't mean I know it is?

No one *knew* it was gathering training data. There was speculation, but as I and chuck posted, it was guesswork because Tesla and Elon never mentioned or confirmed it.

In the context of a million car fleet, it still doesn't make sense for them to hire manual drivers for data. It really only makes sense in context of the 0mph stops.

 

[uscbucsfan]

Yea I don't know why this is so difficult to understand. Perhaps during testing, the driver had to disengage and complete the maneuver manually?

No one *knew* it was gathering training data. There was speculation, but as I and chuck posted, it was guesswork because Tesla and Elon never mentioned or confirmed it.

Yes, but as almost every here tried to explain to you...it was the obvious conclusion. You were bent on V12 doesn't need individual training, which was and is incorrect.

 

[JB47394]

Uh huh, and nothing I said there was wrong

You couldn't have been more wrong.

If you two want to get pissy with each other, could you do it with private messages?

 

[zoomer0056]

Yea I don't know why this is so difficult to understand.

Yeah, just a nit:

 

[Daniel in SD]

The TSLA ADAS drivers once again at Chuck's unprotected left turn today. Is the end to end net thick as a brick? Posted about 10 minutes ago.

View attachment 1027403

No, they're going to finish the "march of nines" and start a limited robotaxi service in Jacksonville (From Chuck's house to the Target).
This will fulfill the promise that V12 "won't be beta" (on one highway in Florida).

 

[EVNow]

There's no more concept of overfitting wrt to V12, overfitting was regarding the heuristics in V11, where the engineers were more biased towards heuristics they were familiar with (Bay Area, CA)

Even with NN there can certainly be overfitting, over sampling etc. Infact they have to oversample to get enough representation for difficult and edge cases - otherwise NN will ignore those.

There is a lot of hard work and judgement that goes into selecting training data for any NN.

 

[powertoold]

Even with NN there can certainly be overfitting, over sampling etc. Infact they have to oversample to get enough representation for difficult and edge cases - otherwise NN will ignore those.

There is a lot of hard work and judgement that goes into selecting training data for any NN.

You're right. There can be overfitting wrt data curation: how many examples of each, etc.

Even using hired drivers for Chuck's UPL is an example of possible overfitting.

I was mostly referring to the idea of overfitting wrt V11 heuristics that were most familiar with the engineers in the Bay Area (when the idea of overfitting was first brought up in that context).

 

[EVNow]

I was mostly referring to the idea of overfitting wrt V11 heuristics that were most familiar with the engineers in the Bay Area (when the idea of overfitting was first brought up in that context).

Yes, with heuristics its a lot more common. With NN the more common thing is perpetuating stereotypes (which Gemini tried to correct .... but they overcorrected).

 

[powertoold]

Yes, with heuristics its a lot more common. With NN the more common thing is perpetuating stereotypes (which Gemini tried to correct .... but they overcorrected).

BTW what's your assessment of V12 right now? Does it look promising for geofenced robotaxis within 2 years, once it gains ability to reverse?

 

[mongo]

Even with NN there can certainly be overfitting, over sampling etc. Infact they have to oversample to get enough representation for difficult and edge cases - otherwise NN will ignore those.

There is a lot of hard work and judgement that goes into selecting training data for any NN.

Indeed...
On the other hand, the testing data set has no such issues

 

[EVNow]

Indeed...
On the other hand, the testing data set has no such issues

I've a lot of experience with getting/generating testing data (for non-NN). It is a very hard problem because you want to include as many edge cases as possible while focusing on the "normal" cases. I'm sure even with NN you need to put in the same kind of hard work and judgement curating test data as you would do with training data. This is where all the data uploaded after disengagements may come in handy.

BTW what's your assessment of V12 right now? Does it look promising for geofenced robotaxis within 2 years, once it gains ability to reverse?

I think it remains difficult to figure out what the rate of improvement will be and will there be plateaus along the way - and how could Tesla get over them. Until now with every approach, Tesla has had slow rate of improvement and eventual plateau. They overcome that by .... rewriting ! So, what can Tesla possibly do this time ?

It was anticipated that end to end NN would make the drive more "natural" and apparently it has (though we need to actually see it in our cars to make definitive statements). But it was also anticipated that it could make sudden dangerous moves - unlike heuristics. That seems to be the case. So - where does that leave us ? Will they be able to add training data to correct mistakes but that doesn't adversely affect other parts of driving ? How long will that take for each problem ?

Keeping all this in mind - I think by the end of the decade Tesla could be close to getting L3/L4 kind of driving in large geofenced areas. But it is equally likely they will be still struggling with say one dangerous move every 1,000 miles ....

 

[mongo]

I've a lot of experience with getting/generating testing data (for non-NN). It is a very hard problem because you want to include as many edge cases as possible while focusing on the "normal" cases. I'm sure even with NN you need to put in the same kind of hard work and judgement curating test data as you would do with training data. This is where all the data uploaded after disengagements may come in handy.

Sure, but in the non-NN case you have to manually create the test cases so need to prioritize what you cover.
For Tesla, they can add an arbitrary number of successful driving clips to the test set for validation. They could choose to exclude some test cases from the final figure of merit or weigh them differently, but adding more of similar situations doesn't result in an over trained network.

 

[drtimhill]

The TSLA ADAS drivers once again at Chuck's unprotected left turn today. Is the end to end net thick as a brick? Posted about 10 minutes ago.

View attachment 1027403

Technically yes, the NN is “thick as a brick”. NN’s have no intelligence or ability to reason whatsoever, they just emit certain outputs based on certain inputs. What makes them interesting is the relative complexity of the inputs and outputs, and the ability construct the transform function via training mechanisms rather then hand-written software.

 

[JB47394]

Technically yes, the NN is “thick as a brick”. NN’s have no intelligence or ability to reason whatsoever, they just emit certain outputs based on certain inputs. What makes them interesting is the relative complexity of the inputs and outputs, and the ability construct the transform function via training mechanisms rather then hand-written software.

My immediate reaction to that is to wonder what the upper limit of a "thick as a brick" system is. What happens when such a system has a quadrillion neurons and all the needed resources to make them work? Can it tackle higher order tasks without "intelligence" and complete them purely by "instinct"? That would make them a kind of "idiot savant", to use the archaic term.

 

[EVNow]

Sure, but in the non-NN case you have to manually create the test cases so need to prioritize what you cover.
For Tesla, they can add an arbitrary number of successful driving clips to the test set for validation. They could choose to exclude some test cases from the final figure of merit or weigh them differently, but adding more of similar situations doesn't result in an over trained network.

If part of NN learning is by minimizing errors in test as well - you have to have fairly representative test data including oversampled edge cases. You are always looking at quantitative figures with testing (% errors etc) which can all be skewed by the kind of (and amount of) test data that you have. I don't think there are any free lunches when it comes to test data either.

 

[kabin]

Palantir CEO Karp interviewed today on CNBC hit the mark on FSD progress for me. Admittedly he mostly talks about LLMs. In so many words he said today, software isn't a luxury product instead it has to work. The assumptions that your product could be non-robust, barely work, and thin was a silicon valley playbook pushed onto the commercial space and it completely failed.

 

[kabin]

Technically yes, the NN is “thick as a brick”. NN’s have no intelligence or ability to reason whatsoever, they just emit certain outputs based on certain inputs. What makes them interesting is the relative complexity of the inputs and outputs, and the ability construct the transform function via training mechanisms rather then hand-written software.

More specifically, thick as a brick is in reference to the net's ability to retain new training data.

 

[powertoold]

BTW, I'm pretty sure TeslaAiGirl was using 12.3 on HW4, based on the smoothed out interior camera housing. HW3 has a more crescenty shaped interior housing:

HW3:

 

[EVNow]

The assumptions that your product could be non-robust, barely work, and thin was a silicon valley playbook pushed onto the commercial space and it completely failed.

I don't know what he means by "commercial space". Is he talking consumer (i.e. retail) or corporate customers ? Corporate software has always been less robust and more error prone than retail/consumer space.

 

[EVNow]

NN’s have no intelligence or ability to reason whatsoever, they just emit certain outputs based on certain inputs.

Do we *know* that is not the case with living organisms .... ?