[idea for] Vision only+personal HD Maps, custom routes?

[JulienW]

No doubt Tesla is correct that Vision only based driving is a much better solution than HD Maps since to have full HD Maps of the US would require TB’s of data that would need GBs of updates daily to stay current. BUT let’s look at the way WE (humans) drive. We also use Vision only and figure out drives and routes without HD Map data. HOWEVER in our daily drives and local area we have HD Maps in knowledge. So in effect we are familiar and dive more efficient/better in our "HD Map area" but can still handle driving in unfamiliar areas too, just a bit more like a “tourist”.

One of FSD's biggest annoyances is it drives like a “tourist” even when driving you home for the 1000x since to FSD it is the “first time” it has ever drove this route.

So my CRAZY idea:

Allow your car to build personal HD Maps you drive daily and save to your SSD. Size of your SSD to determine of much HD Map data (and Dashcam) you can have. This way the car would drive soooooo much better and you could even make custom routes/shortcuts that are better suited to you than what the Navigation picks. Also if you get a new Tesla you simply change the SSD and your New Tesla has instantly “learned” your personal driving routes, HD Maps and preferences. Tesla could even offer subscriptions cloud uploads and even offer pay to download HD Maps of a new area you are going to visit.

Seems like this would be the best of BOTH (Vision only and HD Maps) and it more like the way we humans drive. Is this just crazy thinking?

 

[Larry Hutchinson]

Absolutely agree. I’ve often thought that local knowledge should augment the maps. For example, location of speed bumps and storm water channels, proper lanes to enter and exit a gated community.

 

[spacecoin]

So you’re saying:
1) Maps are bad so TV is better without maps.
2) Maps where you drive would be better

Which one is it? Maps good or maps bad?

Crowd sourcing of maps is not a novel idea. Mobileye has done it for 5-6 years or more. These are no HD-maps though. You need better sensing and positioning to build hd-maps than a few cameras. You also need a decent ML flow to handle all the data automatically. Than can’t be done on the car, and you need some human touches here and there or QA.

Waymo build an initial map and keeps it up to date with the fleet and ML. As they have sufficient hardware to to proper mapping on the cars, it’s easier for them

Personally I think maps are great. They are prior that let you plan further ahead and “see” occuluded signs etc. As you say, just like humans work. We drive better where we know what’s ahead. So do machines.

 

[JulienW]

So you’re saying:
1) Maps are bad so TV is better without maps.
2) Maps where you drive would be better

Which one is it? Maps good or maps bad?....

Can you NOT read? Didn't say ANY of that.

 

[spacecoin]

Can you NOT read? Didn't say ANY of that.

You said exactly that though.

No doubt Tesla is correct that Vision only based driving is a much better solution than HD Maps

“Maps are bad”

Seems like this would be the best of BOTH (Vision only and HD Maps) and it more like the way we humans drive. Is this just crazy thinking?

“Maps are good, will improve performance”

I’ve explained in my first post why this isn’t feasible to do on-car and so on.

 

[ewoodrick]

The ironic thing here is that Tesla is doing this. In one of the last meetings, they were discussing how the vehicles collect the information about intersections and that they are collecting it and having it available to use.

BUT, HD maps are NOT the panacea that you may believe. Waymo and Cruise prove this regularly.
That's why they send safety drivers out at the beginning of the day to mark any problematic areas.
That's why one of the cars drove into freshly paved concrete.
That's why a few weeks ago, a bunch got jammed up because construction blocked off a lane and the only way around it was up on the Interstate, somewhere where they aren't allowed to go.

So everywhere where you may have HD maps, you have to assume that they are wrong.

I'm pretty sure that Tesla has as much, if not more detailed road data than Waze. The maps are extremely precise and seem to have every turn lane and road imaginable.

 

[spacecoin]

The ironic thing here is that Tesla is doing this. In one of the last meetings, they were discussing how the vehicles collect the information about intersections and that they are collecting it and having it available to use.

BUT, HD maps are NOT the panacea that you may believe. Waymo and Cruise prove this regularly.
That's why they send safety drivers out at the beginning of the day to mark any problematic areas.
That's why one of the cars drove into freshly paved concrete.
That's why a few weeks ago, a bunch got jammed up because construction blocked off a lane and the only way around it was up on the Interstate, somewhere where they aren't allowed to go.

So everywhere where you may have HD maps, you have to assume that they are wrong.

I'm pretty sure that Tesla has as much, if not more detailed road data than Waze. The maps are extremely precise and seem to have every turn lane and road imaginable.

A few examples of failures doesn't invalidate the idea that maps add safety and other value such as ride quality.

Waymo's chief scientist on mapping: (31:50)

 

[ewoodrick]

A few examples of failures doesn't invalidate the idea that maps add safety and other value such as ride quality.

Waymo's chief scientist on mapping: (31:50)

I think that he said pretty much exactly as I said and as Tesla has done.

I hear him saying that his cars can't read the road that well. And with AI, it could do better. Tesla has been there, done that. And that AI is needed to scale it. Again, Tesla, been there, done that.

Sure, early on, there were times that I wished that the car would just learn from me how to do it right and remember it, but that has been years ago now.

You have to remember that there are still a number of major releases planned before FSD goes into production, each being leaps past the previous. There's still a LOT to come.

V12 is being shown because it finally is a release where it works the vast majority of the time. Sure it has a few issues, it's conservatively safe, therefore being called slow. But that in itself is a big change from previous versions

And I'm sure that one of the bigger reasons that things like this are problematic is liability. If the car learns from you, who is liable.
But of course, the bigger one is what if there is no driver?

 

[JulienW]

The ironic thing here is that Tesla is doing this. In one of the last meetings, they were discussing how the vehicles collect the information about intersections and that they are collecting it and having it available to use.

BUT, HD maps are NOT the panacea that you may believe. Waymo and Cruise prove this regularly.
That's why they send safety drivers out at the beginning of the day to mark any problematic areas.
That's why one of the cars drove into freshly paved concrete.
That's why a few weeks ago, a bunch got jammed up because construction blocked off a lane and the only way around it was up on the Interstate, somewhere where they aren't allowed to go.

So everywhere where you may have HD maps, you have to assume that they are wrong.

I'm pretty sure that Tesla has as much, if not more detailed road data than Waze. The maps are extremely precise and seem to have every turn lane and road imaginable.

In my scenario FSD continues to drive just like it does but it augments your most used routes, like driving to work by building and maintaining an HD Map to supplement its Vision system. So this is an addition to Vision FSD and NOT a replacement. This way it can "learn" all your street's nuances and "remember" details just like you do instead of encountering it for the "first time" every day. Agian this is not selective FSD/NN/Mothership learning since the HD Maps would be saved on your SSD. So remove the SSD and it drives exactly like every other Tesla. Put your SSD in a loaner Tesla and you have your preferred routes and HD Maps of your daily drives.

 

[gsmith123]

Crowd sourcing of maps is not a novel idea. Mobileye has done it for 5-6 years or more. These are no HD-maps though. You need better sensing and positioning to build hd-maps than a few cameras. You also need a decent ML flow to handle all the data automatically. Than can’t be done on the car, and you need some human touches here and there or QA.

It probably depends on what exactly one has in mind for an HD map. Which brings me to ask:

What did you all have in mind for what information should be mapped on this HD map? That seems like a pretty big question that I don't seen any discussion of yet. There's a pretty broad range of possibilities here, from basic lane topology and sign positions (stop signs, yields, speed limits, etc) on the easier end, to more granular data like cm level mapping of road markings or extremely precise position data for pot holes.

 

[dtdtdt]

So my CRAZY idea:

Allow your car to build personal HD Maps you drive daily and save to your SSD. Size of your SSD to determine of much HD Map data (and Dashcam) you can have. This way the car would drive soooooo much better and you could even make custom routes/shortcuts that are better suited to you than what the Navigation picks. Also if you get a new Tesla you simply change the SSD and your New Tesla has instantly “learned” your personal driving routes, HD Maps and preferences. Tesla could even offer subscriptions cloud uploads and even offer pay to download HD Maps of a new area you are going to visit.

You might be interested in a less data-intensive proposal: Adequate Self Driving. Basically, Tesla should allow you to add custom map overrides. Tesla already supports some of this when you set a route and the car downloads additional info about the route from the mother ship.

 

[spacecoin]

I think one needs to consider how auto regulation works (eg NHTSA), how insurance works, how manufacturer liability works etc before suggesting hobby-level suggestions. The solution is clearly not to have the customers "mod" their cars' behaviour like described in Adequate Self Driving.

If you need maps to improve driving and safety, just equip the cars with a proper 5G connection and have it download any level of map data tiles needed to improve the driving in advance based on location and on a just in time basis as needed and cache them. It's really not a technical problem.

If maps are truly needed for autonomous driving or even good driving, and on what level of detail they should be, that's for the future to tell.

What is a fact though is that everyone that has deployed autonomous vehicles use high(er) definition maps (and many other things) that Tesla doesn't.

 

[JulienW]

Guys in my example the HD Maps are MADE by your car, KEEP on your SSD and UPDATED every time you drive on your HD Mapped streets by your car. No other cars, data or HD Maps sources needed for your personal HD Maps. Your HD Maps are unique to you, your personal property and not NEEDED in any way. They are just your personal augmentation to improve FSD driving on your most used routes like driving to work.

This is (I believe) the same basic way Waymo/Cruse build HD Maps except they also exchange data between cars driving in the same geofenced area. There is NO exchange of data needed in my scenario since your car drives these same streets daily/weekly.

So your HD Maps built/maintained on your SSD are your property and exclusive to you. Any Tesla you drive you can use your HD Maps by plugging in your SSD or not use and it still drives the same.

So no Tesla car is any different or knows any more or less about driving anywhere than any other Tesla car unless you plug in your SSD with your personal HD Map data.

Above is what I would like to see - Below is ideas of additional services Tesla could offer

[an OPTIONAL opportunity for Tesla revenue] In my original post Tesla COULD also offer HD Map services based on this like offer to store your personal HD Map data in the cloud (like Driver Profile but change a fee) and they could (not required but an opportunity for revenue) also build (since they are allowed access to all the data your car collects) and offer specialty HD Maps of select cities or locations and charge a usage fee to download. EXAMPLE: Live in ATL and going to Charlotte for the business trip. I could buy/rent the Downtown (oddly Charlotte calls it Uptown) HD Map area so my Tesla will drive like a "local" or not buy it and it drives like a "tourist" the exact same as now.

 

[spacecoin]

You main thesis seem to be that HD-maps can improve performance. I think that's likely too.

If maps meaningfully improves the performance of the ADAS/TaaS-solution that Tesla lets you subscribe to, then they should add the necessary hardware to the cars to collect and maintain such map data and perhaps have different tiers for it (eg L2+ only, and L2+ limited and L3 on highways for example, where the latter requires HD-maps in this example). Perhaps the HD-maps feature world requires a car manufactured after 2025 with additional sensors and a 5G connection. I doubt the current models are capable of it, considering storage, compute, connectivity and sensing.

I don't understand why you think this problem need customer involvement and hard drives etc to be solved. Why wouldn't Tesla want to own and process the map data if the car is capable of collecting it? It's a valuable asset that you can base services on, like selling information to infrastructure owners about damaged signs, broken traffic lights, and potholes etc.

Furthermore as I wrote in the post above, ADAS and AV-systems are subject to regulation and using customer provided equipment for it is not a good idea.

 

[ewoodrick]

I think one needs to consider how auto regulation works (eg NHTSA), how insurance works, how manufacturer liability works etc before suggesting hobby-level suggestions. The solution is clearly not to have the customers "mod" their cars' behaviour like described in Adequate Self Driving.

If you need maps to improve driving and safety, just equip the cars with a proper 5G connection and have it download any level of map data tiles needed to improve the driving in advance based on location and on a just in time basis as needed and cache them. It's really not a technical problem.

If maps are truly needed for autonomous driving or even good driving, and on what level of detail they should be, that's for the future to tell.

What is a fact though is that everyone that has deployed autonomous vehicles use high(er) definition maps (and many other things) that Tesla doesn't.

I'm not really disagreeing with you, but you kind of brought up the reason why Tesla has two sets of maps.

The map segments that you see (Google Maps) are indeed downloaded and displayed for you to see. They are downloaded as needed and you can sometimes see when the maps are missing.
But the maps that the car downloads for itself (OpenSourceMaps) includes possibly the world and everything is downloaded at once. This allows the vehicles to be in areas without cellular coverage and still operate. It's not downloaded on-the-fly, it's downloaded a few times a year in its entirety.

If you look in the car at the software version that it the car is running, near it is the map version and download date.

 

[spacecoin]

I'm not really disagreeing with you, but you kind of brought up the reason why Tesla has two sets of maps.

The map segments that you see (Google Maps) are indeed downloaded and displayed for you to see. They are downloaded as needed and you can sometimes see when the maps are missing.
But the maps that the car downloads for itself (OpenSourceMaps) includes possibly the world and everything is downloaded at once. This allows the vehicles to be in areas without cellular coverage and still operate. It's not downloaded on-the-fly, it's downloaded a few times a year in its entirety.

If you look in the car at the software version that it the car is running, near it is the map version and download date.

Yes, I am aware of how Tesla currently works with maps for driving and navigation respectively.

 

[JulienW]

You main thesis seem to be that HD-maps can improve performance. I think that's likely too.

If maps meaningfully improves the performance of the ADAS/TaaS-solution that Tesla lets you subscribe to, then they should add the necessary hardware to the cars to collect and maintain such map data and perhaps have different tiers for it .....Perhaps the HD-maps feature world requires a car manufactured after 2025 with additional sensors and a 5G connection. I doubt the current models are capable of it, considering storage, compute, connectivity and sensing......

FIRST: Should be renamed Tesla Vision Maps since HD Maps is ambiguous and has a negative connotation because of Elon's statements.

There is no need for any additional hardware and even NO data connection. All personal Vision Mapping would be done by your car's current camera system and saved on your SSD.

The reason for the SSD storage is 3 fold.

• Don't want any cars to be "smarter" or "better" FSD drivers than any other, so sans SSD they have identical capabilities
• Most important the Vision Maps could require many GBs of storage and that is not available in our cars
• Being on your SSD you can easily remove it for any reason like moving to another Tesla or copying for significant other

 

[BitJam]

I'm not really disagreeing with you, but you kind of brought up the reason why Tesla has two sets of maps.

The map segments that you see (Google Maps) are indeed downloaded and displayed for you to see. They are downloaded as needed and you can sometimes see when the maps are missing.
But the maps that the car downloads for itself (OpenSourceMaps) includes possibly the world and everything is downloaded at once. This allows the vehicles to be in areas without cellular coverage and still operate. It's not downloaded on-the-fly, it's downloaded a few times a year in its entirety.

If you look in the car at the software version that it the car is running, near it is the map version and download date.

It has been shown many times here that the navigation maps are not downloaded all at once. People who don't plan ahead and drive long distances out of cell range end up with no navigation data for their current location. The solution is to browse the area you plan to travel to while you still have cell reception.

On a related note, two weeks ago the navigation map for my part of rural Mew Mexico got significantly better. My car finally knows how to get to the grocery store which should make FSD less useless and less frustrating. But there was no firmware update, I'm still on 2024.8.9, and the software tab says it's still on navigation data from 2023. I had noticed other improvements to the nav data which caused me to see if it could now get to the grocery store.

Clearly Tesla has a mechanism for updating the nav data on the fly (unless the date of the nav data displayed in the software tab is broken).

 

[diplomat33]

No doubt Tesla is correct that Vision only based driving is a much better solution than HD Maps since to have full HD Maps of the US would require TB’s of data that would need GBs of updates daily to stay current. BUT let’s look at the way WE (humans) drive. We also use Vision only and figure out drives and routes without HD Map data. HOWEVER in our daily drives and local area we have HD Maps in knowledge. So in effect we are familiar and dive more efficient/better in our "HD Map area" but can still handle driving in unfamiliar areas too, just a bit more like a “tourist”.

One of FSD's biggest annoyances is it drives like a “tourist” even when driving you home for the 1000x since to FSD it is the “first time” it has ever drove this route.

So my CRAZY idea:

Allow your car to build personal HD Maps you drive daily and save to your SSD. Size of your SSD to determine of much HD Map data (and Dashcam) you can have. This way the car would drive soooooo much better and you could even make custom routes/shortcuts that are better suited to you than what the Navigation picks. Also if you get a new Tesla you simply change the SSD and your New Tesla has instantly “learned” your personal driving routes, HD Maps and preferences. Tesla could even offer subscriptions cloud uploads and even offer pay to download HD Maps of a new area you are going to visit.

Seems like this would be the best of BOTH (Vision only and HD Maps) and it more like the way we humans drive. Is this just crazy thinking?

Honestly, I am not really sure how this would be any different from using the fleet to crowdsource maps. I mean if every Tesla is going to do this for their own personal routes, why not just upload it to the cloud and have crowdsourced maps that everybody can use? You even mention that people could pay to download maps for new areas. So it is basically the same as a crowdsourced map.

I have often wondered why Tesla does not do this. It would seem like an obvious thing and would have many benefits. I know I have had many instances where I wish I could teach my car to take a certain route or do drive a certain way and wish it could remember it.

 

[enemji]

No doubt Tesla is correct that Vision only based driving is a much better solution than HD Maps since to have full HD Maps of the US would require TB’s of data that would need GBs of updates daily to stay current. BUT let’s look at the way WE (humans) drive. We also use Vision only and figure out drives and routes without HD Map data. HOWEVER in our daily drives and local area we have HD Maps in knowledge. So in effect we are familiar and dive more efficient/better in our "HD Map area" but can still handle driving in unfamiliar areas too, just a bit more like a “tourist”.

One of FSD's biggest annoyances is it drives like a “tourist” even when driving you home for the 1000x since to FSD it is the “first time” it has ever drove this route.

I have written about localization, which is essentially targeted to achieve the same/similar end result, and I believe 100% that localization is happening. I have many roads and intersections that are handled liked a tourist after every upgrade but within a weeks time, the FSD is able to handle it like it knows the road like a local. This requires very little data and hence nobody is actually able to track it looking at the data bank sizes which are in TB