OFFICIAL BUTTON WATCH

[EVNow]

It is not useless. It still tells you that you don't need a driver for the ODD, in contrast with L0-L3 that do require a driver. But the SAE is very clear that the level alone is not sufficient to describe the entire performance of the autonomous driving and that you need to also specify the ODD (unless L0 or L5):

It’s useless. Without defining some kind of minimum ODD, it’s not useful.

In the companies I’ve worked something vague like this would not be approved as a standard.

 

[diplomat33]

It’s useless. Without defining some kind of minimum ODD, it’s not useful.

In the companies I’ve worked something vague like this would not be approved as a standard.

You do have to define the ODD. But the ODD is defined separately from the levels.

 

[Bladerskb]

I don't know if it is "worthless". That's a bit strong. The data does help Tesla train their NN. But certainly, the real world data is not the decisive advantage that Tesla fans believe it to be. That's because, correct me if I am wrong, but it seems to me that quality is more important than quantity after a certain point. Just having a lot of data does not necessarily translate into more progress. There is no 1:1 correlation between number of miles and rate of FSD progress.

Also, don't know if you saw this but Elon admitted that they overfit to the Bay Area:

So Tesla might have "billions of miles" of data but most of the data is confined to areas with high density of Teslas, so it is really just working great in some areas where they overfit. Other areas, where Tesla has fewer cars and less data, FSD does worse.

Nah i disagree. It is essentially worthless because at some point after 10k-50k cars, you start getting diminishing returns on data and the bottleneck becomes something else.
This is why not every Tesla car gets a campaign plus also Tesla trains with hundreds of millions of images and not hundreds of billions.

Think about it. You have 50k cars, which gives you about 1,500,000 miles per day which is more than enough to find whatever needle in the haystack your data engine is looking for. Actually its too much data. Remember the bottlenecks all exist elsewhere. Those 1.5 Million miles of day, you are not even using nor will you keep.

The overfit Elon is talking about is pertaining to the driving policy. Their engineers are in CA so that's where they develop and test the driving policy in house. Even though there are what 50% of teslas in CA. It doesn't affect the amount of data they can get from the other 50%.

So there's no overfit happening in the perception system, only in the planning and control which is 100% hand-engineered at Tesla.

 

[diplomat33]

Nah i disagree. It is essentially worthless because at some point after 10k-50k cars, you start getting diminishing returns on data and the bottleneck becomes something else.
This is why not every Tesla car gets a campaign plus also Tesla trains with hundreds of millions of images and not hundreds of billions.

Think about it. You have 50k cars, which gives you about 1,500,000 miles per day which is more than enough to find whatever needle in the haystack your data engine is looking for. Actually its too much data. Remember the bottlenecks all exist elsewhere. Those 1.5 Million miles of day, you are not even using nor will you keep.

Thanks. I don't think we are that far off. Like I said, big data is not the big advantage that Tesla fans claim it is, because you want quality over quantity. I think that is similar to what you are saying.

The overfit Elon is talking about is pertaining to the driving policy. Their engineers are in CA so that's where they develop and test the driving policy in house. Even though there are what 50% of teslas in CA. It doesn't affect the amount of data they can get from the other 50%.

So there's no overfit happening in the perception system, only in the planning and control which is 100% hand-engineered at Tesla.

Thanks. I had not thought about overfit for planning.

I don't think it is possible to solve L5 with planning that is done by hand. How are they going to do the driving policy for all the areas outside of CA, by hand? It seems to me that they will need to go to ML planning which will require another rewrite and of course, collecting data from that other 50% of the fleet to properly train the planning for all the areas outside of CA which is a very large area. This will take time.

 

[EVNow]

Think about it. You have 50k cars, which gives you about 1,500,000 miles per day which is more than enough to find whatever needle in the haystack your data engine is looking for. Actually its too much data.

We don't know that.

I've worked in companies whose products where used by Billions (yes "B"). There were always surprises and edge cases to be found. There was no such thing "too much data". You just have to be good at filtering.

 

[EVNow]

You do have to define the ODD. But the ODD is defined separately from the levels.

Where is the ODD standard defined that the levels standard refers to ? If there is no other standard - then levels standard has to define it.

The levels "standard" is just hand-waving at this point.

 

[diplomat33]

Where is the ODD standard defined that the levels standard refers to ? If there is no other standard - then levels standard has to define it.

The levels "standard" is just hand-waving at this point.

J3016 does define ODD:

Operating conditions under which a given driving automation system or feature thereof is specifically designed to function, including, but not limited to, environmental, geographical, and time-of-day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics.

But it up to the manufacturer to specify the ODD. You can't include the ODD into the levels themselves. They have to be separate.

 

[powertoold]

We don't know that.

I've worked in companies whose products where used by Billions (yes "B"). There were always surprises and edge cases to be found. There was no such thing "too much data". You just have to be good at filtering.

Actually, we do know that Tesla could use *more* data. Even Elon talks about having to simulate certain crashes because there aren't enough examples of them.

We know that Teslas crash about once every 3-4 million miles, according to Tesla. There's around 500k+ of the hw3 cars in the USA, so there's only 500k * 20 miles / 3,000,000 miles = 3.33 crashes a day, which aren't that many if you want a lot of crash data.

 

[Daniel in SD]

Actually, we do know that Tesla could use *more* data. Even Elon talks about having to simulate certain crashes because there aren't enough examples of them.

We know that Teslas crash about once every 3-4 million miles, according to Tesla. There's around 500k+ of the hw3 cars in the USA, so there's only 500k * 20 miles / 3,000,000 miles = 3.33 crashes a day, which aren't that many if you want a lot of crash data.

Maybe FSD Beta will solve this problem?

 

[idriveacar]

How exactly is he wrong? He is absolutely right for other reasons and that reason being that after about 6 years of AP development which is now going on 7 years. They still are no where close to a safe L5 system. So what happened to the billions of miles of data advantage? Why hasn't it shown up? According to ArkInvest which is regarded as gods in Tesla circles. They say that Tesla has over 10 billion miles of data compared to Waymo's 10 million. Yet where is the L4 let alone L5?

The cool thing about Tesla's data advantage is that it doesn't disappear when they retrain their NNs. Their data pipeline has undergone massive improvements over the last 4 years. Every time they modify the architecture, they "retrain" using that same data they've collected, plus whatever new data they have (and potentially minus some data, if they want).

And they're working on improving their data set constantly.

"Retrain" in the context of NNs isn't "go back to square one" nor does it mean "rewrite" or anything like that. It's simply a normal part of the development of ML systems. Their data pipeline is kind of like a factory, their labeled datasets are like the raw materials, and the NNs that get deployed to the cars are like the finished product.

Every time they "retrain", that just means they tweaked the pipeline and have to shovel all the labeled data though it again. I think at AI day they said it takes a few days with their (thousands of GPUs) cluster to retrain the perception NNs.

And finally... visual perception doesn't drive cars. While necessary, it isn't sufficient. There are challenges that Tesla is still ironing out, that aren't solved by just having the best dataset.

But I don't think you're giving Tesla enough credit. FSD seems to be very solid for something that many experts claimed to be impossible (or at least many decades away) 6 years ago. They've made a ton of progress.

 

[Matias]

I don't think it is possible to solve L5 with planning that is done by hand. How are they going to do the driving policy for all the areas outside of CA, by hand? It seems to me that they will need to go to ML planning which will require another rewrite and of course, collecting data from that other 50% of the fleet to properly train the planning for all the areas outside of CA which is a very large area. This will take time.

I also believe that they specifically overfit driving policy to Elon‘s daily commute.

 

[caligula666]

well, I’m not planning to drive from the Kansas to Tokyo so who gives a flying ****. This point reeks of desperation.

Well, I don’t quite care where you want to drive or not the point is that Tesla sold cars outside of the USA.

 

[diplomat33]

I also believe that they specifically overfit driving policy to Elon‘s daily commute.

Gotta keep the boss happy.

 

[EVNow]

I also believe that they specifically overfit driving policy to Elon‘s daily commute.

They probably overfit for the personal driving routes of all the testers - including Musk.

Given how much people here love to bash Musk - its no wonder I don't see many complement him on his brutal honesty in that tweet (which praised FSD !).

 

[mcristina]

I think it will definitely come this year, looking at all the progress made we're getting close

 

[powertoold]

I think it will definitely come this year, looking at all the progress made we're getting close

I'm highly confident that The Button (or some form of fsd beta wide release in the USA) will come this year. This is not a question mark; 3 months maybe, less than 6 months definitely, but more than probably two weeks.

 

[diplomat33]

I'm highly confident that The Button (or some form of fsd beta wide release in the USA) will come this year. This is not a question mark; 3 months maybe, less than 6 months definitely, but more than probably two weeks.

Is that you Elon?

 

[planetary]

I don't think it's this year. However, for the first time I've got a car on order with FSD equipped...

 

[powertoold]

You sound like Elon.

I'm actually not that excited about The Button (other than the fact that it would represent a major leap in fsd progress). I had hoped they would include reverse summon along with fsd beta. That way, I can have fsd beta get to my destination and also park for me. It'd be the ultimate fsd satisfaction (if no interventions required).

 

[diplomat33]

I'm actually not that excited about The Button. I had hoped they would include reverse summon along with fsd beta. That way, I can have fsd beta get to my destination and also park for me. It'd be the ultimate fsd satisfaction (if no interventions required).

V10 will include retraining per Elon's tweet. So I think there is a chance it could also include reverse summon. And we know it will be a single stack for both highway and city, it would be a great surprise if it also included reverse summon in order to be "feature complete". And I think Elon wants to deploy "feature complete" as soon as possible. So it is possible that Elon will want reverse summon in V10 too. It just depends if it is ready or not.

I am actually looking forward to the Button a lot. I know it will require driver supervision and there will be interventions but I am eager to try FSD Beta for myself. And having "end to end" would be nice. No more disengaging all the time, because I need to make a turn. Also, I am really curious how FSD Beta handles the roads on the college campus where I work. There are some interesting edge cases. I might even do a video of it.