Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Autonomous Car Progress

eli_

Member
May 14, 2018
454
982
Bellevue, WA
What's becoming evident is that HD maps are leading to a local maximum. It's been how many years since they were first used?

HD maps are very tempting to use, for the very reasons we see 10.2 fail. But, Elon and everyone keeps repeating the same thing, which is that vision needs to be "solved" before FSD can be achieved. Basically, vision is the rate-limiting factor for FSD development, not HD maps. Once vision is adequate for multiples of human safety, then HD maps aren't useful, they'd just be more noise.
Tesla's BEVnet that predicts roadway geometry from vision is essentially a global HD map in a compressed format. At least when it's fully trained and labeled from global roads. Instead of storing explicit, human-understandable maps, they are encoded as weights in a network, and the maps are unpacked out of the network when you input camera frames instead of GPS coordinates. This has the benefit of working entirely offline, doesn't require precise localization (or any localization), and it doesn't totally fall apart when it encounters an unmapped roadway or stale map data.
 

JHCCAZ

Member
Supporting Member
Feb 2, 2021
430
750
Tucson
Tesla's BEVnet that predicts roadway geometry from vision is essentially a global HD map in a compressed format. At least when it's fully trained and labeled from global roads. Instead of storing explicit, human-understandable maps, they are encoded as weights in a network, and the maps are unpacked out of the network when you input camera frames instead of GPS coordinates. This has the benefit of working entirely offline, doesn't require precise localization (or any localization), and it doesn't totally fall apart when it encounters an unmapped roadway or stale map data.
Andrej Karpathy discussed this a bit in the AI day presentation, mostly as a future state of affairs. It's an understandable concept and kind of exciting to think that the car will be familiar with all roads as long as a few Tesla drivers have been there before.

So I'm fully on board with the concept and looking forward to its realization. However, so far I think there is little evidence that oft-driven routes are making their way into the NN, encoded in the gigantic matrices of weighting coefficients. Certainly we see the FSD beta testers encountering the same route, intersection and lane confusions, release after release, with some improvements but also with fresh pathologies. Even the much-discussed California-rich training advantage is not clearly evident regarding routes and lanes - it comes probably more from a better understanding of California style road design and markings.

At the other end of the spectrum are the North Carolina test videos put out by Rocco Speranza. Compared to California, NC is a nearly Tesla-free zone and there are many unfortunate errors of planning, routing and understanding. For a while he's been chalking those up largely to navigation-map problems and I think that's right.

One other note - when it does happen, I think it still is not within the definition of "HD Maps". Those are specifically centimeter-resolution databases and include an enormous amount of labeled objects, road paintings and everything that doesn't normally move. When the NN-memory map does arrive, it it won't be a so-called HD map, but a huge collection of "I've been here" memory engrams (a great word I learned as a kid by watching the original Star Trek series, long before artificial brains were anything close to achievable).

Here's a very relevant, entertaining and classic song from that same era, this cover by the great Johnny Cash (done earlier by Hank Snow, and there's a great UK version). This is how Tesla will hopefully be in a few years:
 
Last edited:
  • Like
Reactions: powertoold

stopcrazypp

Well-Known Member
Dec 8, 2007
11,132
6,268
Andrej Karpathy discussed this a bit in the AI day presentation, mostly as a future state of affairs. It's an understandable concept and kind of exciting to think that the car will be familiar with all roads as long as a few Tesla drivers have been there before.

So I'm fully on board with the concept and looking forward to its realization. However, so far I think there is little evidence that oft-driven routes are making their way into the NN, encoded in the gigantic matrices of weighting coefficients. Certainly we see the FSD beta testers encountering the same route, intersection and lane confusions, release after release, with some improvements but also with fresh pathologies. Even the much-discussed California-rich training advantage is not clearly evident regarding routes and lanes - it comes probably more from a better understanding of California style road design and markings.

At the other end of the spectrum are the North Carolina test videos put out by Rocco Speranza. Compared to California, NC is a nearly Tesla-free zone and there are many unfortunate errors of planning, routing and understanding. For a while he's been chalking those up largely to navigation-map problems and I think that's right.

One other note - when it does happen, I think it still is not within the definition of "HD Maps". Those are specifically centimeter-resolution databases and include an enormous amount of labeled objects, road paintings and everything that doesn't normally move. When the NN-memory map does arrive, it it won't be a so-called HD map, but a huge collection of "I've been here" memory engrams (a great word I learned as a kid by watching the original Star Trek series, long before artificial brains were anything close to achievable).

Here's a very relevant, entertaining and classic song from that same era, this cover by the great Johnny Cash (done earlier by Hank Snow, and there's a great UK version). This is how Tesla will hopefully be in a few years:
Yep, this was mentioned on AI day, but it's a very abstract idea. It's basically a "map" in the sense of a AI that have driven enough in a variety of roads, that just from looking at a particular road feature, it predicts what is the road structure. But it's definitely not anywhere the same as a map as we typically understand it.

But I guess the difference in this approach is that even though currently the NN's understanding of roads is heavily CA Bay Area biased, even when you drop it into an unfamiliar road, there is some basic competence (for example someone in Ukraine who hacked FSD Beta to activate it).
 
  • Like
Reactions: MP3Mike

powertoold

Active Member
Oct 10, 2014
3,078
6,601
USA
NN's understanding of roads is heavily CA Bay Area biased

Hmm, I may be wrong about this, but I'm pretty sure that the road geometry NN was trained on diverse USA roads, so the road geometry NN itself isn't biased to CA Bay Area.

What is biased to CA Bay Area is the driving policy and planning. Basically, preception = USA wide, planning and policy = overfit to Bay Area since that's where most of the engineers are driving their cars. Just my interpretation about what's going on.

Elon's wording is deceptive because "overfit" is often used in relation to NN training to mean that the outputs aren't good at generalizing outside the dataset. But in this case, I think he just means the procedural code that is used for policy and planning is biased towards the Bay Area.

 
Last edited:
  • Like
Reactions: EVNow

stopcrazypp

Well-Known Member
Dec 8, 2007
11,132
6,268
Hmm, I may be wrong about this, but I'm pretty sure that the road geometry NN was trained on diverse USA roads, so the road geometry NN itself isn't biased to CA Bay Area.

What is biased to CA Bay Area is the driving policy and planning. Basically, preception = USA wide, planning and policy = overfit to Bay Area since that's where most of the engineers are driving their cars. Just my interpretation about what's going on.

Elon's wording is deceptive because "overfit" is often used in relation to NN training to mean that the outputs aren't good at generalizing outside the dataset. But in this case, I think he just means the procedural code that is used for policy and planning is biased towards the Bay Area.
I'm just referring to the fact from the CA DMV filings a majority of testers are in CA and in other discussions there was evidence present in how an example of an offset left turn (meaning at a left turn, the lanes across the street is a bit off) perception error seems to match CA Bay Area road structure, but is different from other areas.
 

EVNow

Well-Known Member
Sep 5, 2009
11,693
32,342
Seattle, WA
I'm just referring to the fact from the CA DMV filings a majority of testers are in CA and in other discussions there was evidence present in how an example of an offset left turn (meaning at a left turn, the lanes across the street is a bit off) perception error seems to match CA Bay Area road structure, but is different from other areas.
There might be some CA bias in NN given most of the cars are in CA - but most likely the planner is the one biased to Bar Area.
 

powertoold

Active Member
Oct 10, 2014
3,078
6,601
USA
There might be some CA bias in NN given most of the cars are in CA

True, but when Tesla is collecting road geometry and lane data, they're looking for diversity, so the data will come from all over the USA. I doubt they're asking the fleet for mostly CA data, but I can see what you mean.
 

Bladerskb

Senior Software Engineer
Oct 24, 2016
2,554
4,263
Michigan
There's no question that the car needs to use a map for navigation. In fact, it seems to me from watching the videos that some fair portion of problems are caused by incorrect navigation maps. I'm concerned that the dependency on OpenStreetMaps, though it sounds cool to be crowd-sourced and all, is limiting the performance of FSD at the moment (not the only limit of course but an important one). Not only is it doubtful that OpenStreetMaps will ever be reliable enough for the task, it seems very unclear how quickly Tesla will update the map after users make corrections. And how many problems are calls because the corrections are not done correctly?
Stale, Un-secure, In-accurate and Unscalable.
This is why Mobileye's REM doesn't rely on, nor is it built on any other mapping system. Its completely standalone.
So let's not let this particular sub-thread devolve into a pitched battle about Maps Good or Maps Bad. I think the debate is about whether the navigation and path planning relies on very high precision mapping of curbs, lanes and myriad other elements of infrastructure.
The debate is already over. The only one still trying to talk about it are people demonstrating purposeful ignorance.
While it does seem tempting to have the HD map data, it's a legitimate question as to how much priority should be placed on the map versus the perceived scene in real time. It's the familiar question: when there is a conflict between the HD map and what is seen, which do you rely on? If you think this through for a while it becomes much less clear that the HD map is very useful at all.
This is has been debunk a billion times and still perpetuated by the Tesla fanbase. Which demonstrates again the inability to think logically and use reason when talking about anything Tesla related. The same people talking about NN detecting objects with confidence metrics simply can't wrap their head around how there could be a tent in the middle of the road. Which AV system can accurately pinpoint with cm accuracy where a mismatch in its map is and that the system would then rely on the NN output based on its confidence level.

This is literally AV Myth #5 : "Myth: An AV can only use its sensors, or its map. It cannot use both at the same time. If the sensor input disagrees with the map, it is an irreconcilable problem and the AV can’t work anymore."

Here look at a Waymo clearly crashing into a tent killing everyone inside after a mis-math between its HD map and its sensor...
Again the only one still spreading this myth are people demonstrating willful ignorance .
 
Last edited:
  • Like
Reactions: Terminator857

hgmichna

Member
Jun 17, 2020
340
269
Germany
Tesla's FSD obviously does not need HD maps of most roads, as it drives almost everywhere just happily and safely without them. If at all, it might be improved by HD maps of difficult junctions.

Has the thought been mentioned or discussed that HD maps might not be needed everywhere, but only for certain difficult places?

And even there a full HD map might not be needed. Perhaps some formalized hints could do the trick already.
 

mark95476

Active Member
Jun 21, 2020
1,979
1,523
Bay Area CA
The TSLAQ crowd is stuck in looney land and still does mind-numbing mental gymnastics when it comes to Tesla. Tomorrow (10/20) is earnings. Tesla had record deliveries this quarter and I'm sure many TSLAQ disciples are praying that Tesla will finally declare bankwuptcy.

#1 TSLAQ fact: Tesla loses $$$ money on every car they build. They keep building and selling more and more cars every quarter. Therefore, larger losses = bankwuptcy. o_O

Tesla is delivering cars, collecting data, updating their NNs, deploying updating firmware, ...etc... Tesla has the scaling and profit parts figured out. Now they're working on the icing on the cake.

Moving goal posts / adjusting to reality. Atleast they are delivering cars.


"In a rapidly changing world, INFINITI is well ahead of the game."

I've been using AP with my Y for <1.5 years and it's been great on freeways and surface streets in the Bay Area. Note that I navigate myself and only allow the car to follow the road. I watch Bjorn's videos who tests virtually everything out there. Tesla's AP/FSD is probably the best or near the top.

I'm a bit surprised that Tesla is doing so well. Tesla is working on so many things and are neck-on-neck with the pure plays like Mobileye and Waymo. I don't know about "Chinese Tech" other than many are using American technology. Things can change in a hurry, but I don't think Tesla will be standing still. Notice how all of these various companies compares themselves to Tesla?

Is Tesla really ahead or it is illusion that they are ahead? I have been driving M3 since 2018 and while UI and lane keeping has improved.. It still makes different decisions under similar type of situations. That introduces elements of uncertainty and as a driver you have no idea what AI is thinking and you will have to take over to avoid getting into accident.

Looking at 10.2 FSD beta videos.. I feel that similar issues exist and system can’t make decisions 100% of the time with certainty.

I think tech is not ready yet.. may be 5-10 years it will look different with newer tech/processing power etc.

In meanwhile, we will likely to see L4 Robotaxis by other companies in cities around the world. It is also likely that Chinese Tech will crack FSD before US.

Just my two cents…
 
  • Funny
Reactions: Daniel in SD

idriveacar

Member
Jun 30, 2021
82
146
Fremont
Stale, Un-secure, In-accurate and Unscalable.
This is why Mobileye's REM doesn't rely on, nor is it built on any other mapping system. Its completely standalone.

The debate is already over. The only one still trying to talk about it are people demonstrating purposeful ignorance.

This is has been debunk a billion times and still perpetuated by the Tesla fanbase. Which demonstrates again the inability to think logically and use reason when talking about anything Tesla related. The same people talking about NN detecting objects with confidence metrics simply can't wrap their head around how there could be a tent in the middle of the road. Which AV system can accurately pinpoint with cm accuracy where a mismatch in its map is and that the system would then rely on the NN output based on its confidence level.

This is literally AV Myth #5 : "Myth: An AV can only use its sensors, or its map. It cannot use both at the same time. If the sensor input disagrees with the map, it is an irreconcilable problem and the AV can’t work anymore."

Here look at a Waymo clearly crashing into a tent killing everyone inside after a mis-math between its HD map and its sensor...
Again the only one still spreading this myth are people demonstrating willful ignorance .


It does look like it got kinda stuck there. Do you have the full video for this clip? I'm not doubting your point BTW, I'm just curious about this clip in particular.
 

Dutchy Ron

Member
Oct 4, 2021
54
50
Netherlands
Yesterday Elon joined a webmeeting with the top 200 VW managers, VW will copy the Tesla approach and will in-source as much as possible. Already 5k employees of Cariad are working on the IT platform for VW, that number will be 10k in 2025. Tesla is miles ahead to competitors, no doubt. The main question is if the current approach and vision will be the right one for the long term and L3+ …… will L4/5 happen in the next 10 years or only in local robot taxis in big cities? In that case Tesla can improve the current FSD feature until L3, now at L2.2 ;-)
 

eli_

Member
May 14, 2018
454
982
Bellevue, WA
If I was Tesla I'd just be L2 forever and proud of it. Why would anyone want to be L4/5, now you're liable for accidents, what business would want that? Nah just be L2 forever and focus on solving the hard AI problems of self-driving while selling intermediate deliverables to customers along the way.
 

rxlawdude

Active Member
Jul 10, 2015
3,295
2,790
Orange County, CA
If I was Tesla I'd just be L2 forever and proud of it. Why would anyone want to be L4/5, now you're liable for accidents, what business would want that? Nah just be L2 forever and focus on solving the hard AI problems of self-driving while selling intermediate deliverables to customers along the way.
I think consumers would be perfectly happy for liability to shift to the manufacturer of L4/5 systems. The manufacturers, of course, not so much.

Consumers would still need collision, comprehensive and other insurance outside of the liability coverage, but that would save a significant chunk of consumers' auto insurance.
 

Dan D.

Member
Dec 7, 2020
855
1,061
Vancouver, BC
If I was Tesla I'd just be L2 forever and proud of it. Why would anyone want to be L4/5, now you're liable for accidents, what business would want that? Nah just be L2 forever and focus on solving the hard AI problems of self-driving while selling intermediate deliverables to customers along the way.
I can foresee a time where Level 2 will be changed to driver assist only, in other words you have to actually be steering all the time. The car can nudge you or beep for hazards. The car can control speed.

Anything where the car controls steering might have to be Level 3, in other words the liability shifts to the manufacturer.

Let's say we have millions of Level 2 cars soon from most manufacturers. Most systems are mostly capable of not crashing, except when they're not. Almost everyone is sort-of paying attention. Of course some others are drunk, sleeping, and in the backseat.

If we have bad Level 2 systems that people are abusing it won't be tolerated for long. I can imagine they'll be told to make better systems, make them Level 3 and no more hands-free Level 2.

Just an opinion.
 
  • Like
Reactions: Dutchy Ron

EVNow

Well-Known Member
Sep 5, 2009
11,693
32,342
Seattle, WA
If we have bad Level 2 systems that people are abusing it won't be tolerated for long. I can imagine they'll be told to make better systems, make them Level 3 and no more hands-free Level 2.
By who ? Will never happen in US, atleast. Regulators are always nice to manufacturers - they know which side of the bread is buttered.
 
  • Disagree
Reactions: Daniel in SD

DarkandStormy

Member
Apr 16, 2021
272
596
Colorado

Steven Cliff, who has been deputy administrator since February, has been a key figure in the Biden administration's proposed rewrite of fuel economy standards through 2026 and is overseeing the department's ongoing safety probe of Tesla Inc and investigation of whether 30 million vehicles produced by nearly two dozen automakers have unsafe air bags.
The White House also plans to announce Duke University engineering and computer science professor Missy Cummings as the new senior adviser for safety at NHTSA.

Take a look at Ms. Cummings twitter feed to get a sense of how she thinks of Tesla (and other AV companies). Not sure if it belongs in this thread, but an interesting choice.
 
  • Informative
Reactions: Dan D.

Products we're discussing on TMC...

About Us

Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.

Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


SUPPORT TMC
Top