how does the new end to end FSD work, need a block diagram from of data flow from the fleet to DoJO to an individual's car

[ewoodrick]

It could drive without maps but Waymo says that HD maps make it drive safer. Thus, Waymo always uses HD maps in order to drive safer.



For validation purposes. "Working everywhere" and "working with 99.9999% reliability" are two very different things. Waymo has said that they dropped the Waymo Driver in LA the first time and it drove very well right out of the gate. But driving very well is not good enough. To launch driverless robotaxis you need better than "drive very well", you need "drive with 99.9999% reliability".



No. The Perception stack uses info from the HD map but does not require HD maps. Using something and requiring it are not the same thing. And saying that the NN requires significant amount of hints is speculation on your part. Again, the reason Waymo uses HD maps is for safety not because the NN can't work without them.

I think there's a real disservice to HD map terminology when used to talk about Waymo or Cruise. Sure, they are HD maps, but I'm pretty sure that they are highly augmented with hints. Satellite systems can create the HD maps.

I'm sorry, you can't say that the NN doesn't need the HD maps and then turn around say that it is used for safety.

These systems will not operate by themselves. They are monitored and fed with data on a continuous basis.

 

[JulienW]

I think there's a real disservice to HD map terminology when used to talk about Waymo or Cruise. Sure, they are HD maps, but I'm pretty sure that they are highly augmented with hints. Satellite systems can create the HD maps.

I'm sorry, you can't say that the NN doesn't need the HD maps and then turn around say that it is used for safety.

These systems will not operate by themselves. They are monitored and fed with data on a continuous basis.

HD Maps are NOT satellite maps at all. First you drive the Waymo around and use the sensors to see the road and everything around, then record this info. That is the HD Map. You then send this data to all Waymo cars and when a Waymo drives over the same street if there are any changes or differences this data overwrites and is sent to the fleet as an update to the HD Maps.

 

[JulienW]

Here is a video that explains HD Maps.

 

[ewoodrick]

Here is a video that explains HD Maps.

Great, so it's a Geomeric layer and a Semantic layer.
The Semantic Layer includes lane and road topology, traffic lights, traffic rules, etc. To make planning decisions.

So listening thorough most of the video, the author ends up effectively suggesting that HD maps won't scale unless you build the vision system that Tesla already has running REALTIME in the cars.

I was quite interesting hearing about each of the scaling needs that Tesla has already pretty much solved.

But the video still really didn't address the Cruise/Waymo needs, except that the routes really need to be continually by safety drivers to assure that no changes has occurred to the HD map.

Tesla just draws the HD map as it goes, solving the scaling issue.

 

[JulienW]

There is no way Tesla can have and use HD Maps. To have a semi coverage of HD Maps for North America would likely be 100s of TB of data. Since Waymo's are geofenced they only need an HD Map for the area they cover. Also they have large SSD for the data since they use commercial grade equipment. Tesla would need to add a large (>1TB) SSD and a system of dynamically downloading/deleting "tons" of data if you were driving to different areas. The cell bill for this data could be astronomical and Tesla would have to pass it on.

Tesla uses Map data for things like Stop signs, Red Lights, No turn on Red and temporary construction. But no HD Map data. Tesla doesn't make a HD Map as it goes since there is no memory once it processes what it "sees". Does kinda suck that every time you drive down the street to work it is the "first time" Tesla has ever seen it and must "figure out AGAIN" the same obstacles, obstructions and oddities that you see everyday and know.

 

[JulienW]

Wondering OT but I had a harebrained idea for personal HD Maps. In a nutshell you buy a SSD and plug in. You can recored HD Maps of the streets and roads you drive the most and dependent on the size of your SSD. You can remove the SSD and put in another Tesla meaning they are your HD Maps.

 

[spacecoin]

DOJO is dead and likely makes up only a tiny fraction of the training computers and becoming more legacy by the day. Bet Tesla hasn't acquired any DOJO chips since the initial FAB run.

Tesla seemingly failed at their own super computer sw/hw ("project Dojo, the D1 chip etc) and is using Nvidia more or less for 100% of their training needs. They have started to change the meaning of "Dojo" from "failed internal project" to "our training computers", which sounds better.

 

[ewoodrick]

There is no way Tesla can have and use HD Maps. To have a semi coverage of HD Maps for North America would likely be 100s of TB of data. Since Waymo's are geofenced they only need an HD Map for the area they cover. Also they have large SSD for the data since they use commercial grade equipment. Tesla would need to add a large (>1TB) SSD and a system of dynamically downloading/deleting "tons" of data if you were driving to different areas. The cell bill for this data could be astronomical and Tesla would have to pass it on.

Tesla uses Map data for things like Stop signs, Red Lights, No turn on Red and temporary construction. But no HD Map data. Tesla doesn't make a HD Map as it goes since there is no memory once it processes what it "sees". Does kinda suck that every time you drive down the street to work it is the "first time" Tesla has ever seen it and must "figure out AGAIN" the same obstacles, obstructions and oddities that you see everyday and know.

It doesn't store them, it creates them while driving, using all of the mechanisms mentioned in the video.
You know, like a human does.

 

[enemji]

Imagine this situation:

At an intersection, there is a STOP sign. It has been noted so on the maps data.
A thunderstorm comes along and that sign is now uprooted and fallen down.

I imagine all the human drivers will no longer stop because there is no STOP sign.

How will Waymo react?
How will Tesla FSD react?

 

[JulienW]

It doesn't store them, it creates them while driving, using all of the mechanisms mentioned in the video.
You know, like a human does.

But we do STORE the knowledge and the next time we drive we remember details and DON'T make the keep making the same mistakes over and over and over. So in effect WE drive like HD Maps on our familiar to us roads BUT like FSD ONLY on roads we are new to.

 

[enemji]

But we do STORE the knowledge and the next time we drive we remember details and DON'T make the keep making the same mistakes over and over and over. So in effect WE drive like HD Maps on our familiar to us roads BUT like FSD ONLY on roads we are new to.

I am pretty sure FSD has local cookies on each car to accommodate such nuances

 

[ewoodrick]

I am pretty sure FSD has local cookies on each car to accommodate such nuances

Pretty sure that it doesn't.

 

[ewoodrick]

But we do STORE the knowledge and the next time we drive we remember details and DON'T make the keep making the same mistakes over and over and over. So in effect WE drive like HD Maps on our familiar to us roads BUT like FSD ONLY on roads we are new to.

Yea, FSD doesn't have a memory. I hope that will be a priority soon.
But that comes back to one of Elon's/Tesla's statements many years ago.

Sure, Tesla can make the cross-country autonomous run as a few cars were doing. But that's not a general case.
Tesla doesn't care about specific cases and won't play in that game. Tesla wants to solve the general case.

And as Tesla is getting so close, all the previous distractions are being proved wrong. LIDAR companies are going out of business because LIDAR doesn't solve the FSD problem.

 

[Ben W]

I think there's a real disservice to HD map terminology when used to talk about Waymo or Cruise. Sure, they are HD maps, but I'm pretty sure that they are highly augmented with hints. Satellite systems can create the HD maps.

As pointed out, such HD maps are created locally with e.g. Lidar, not by satellite.

I'm sorry, you can't say that the NN doesn't need the HD maps and then turn around say that it is used for safety.

In FSD's case, the maps can be helpful as a prior, just as your memory of a city layout is helpful when you drive it again. But they are not necessary, and FSD will override the maps when appropriate. (E.g. when a lane is blocked, or if a new stop sign is put in.) FSD actually doesn't use "HD" maps at all, just ordinary street maps.

That said, more sources of information can always be used to enhance safety. I still think Lidar and radar would be valuable additions to the sensor suite, to compensate for Pure Vision's shortcomings in rain/fog/dirt or low light. If Robotaxi without Lidar could be 99.9999% reliable, perhaps Robotaxi with Lidar could be 99.999999% reliable.

These systems will not operate by themselves. They are monitored and fed with data on a continuous basis.

FSD certainly operates by itself. It has ordinary street maps pre-downloaded, which allows it to figure out directions even without a cell signal, and it can operate perfectly well in the absence of any internet connectivity. (I mean, of course it makes mistakes now and then, but it works just as well without as with.) I park in an underground garage at work with no signal, and often it takes 5-10 minutes after leaving the garage before the signal comes back, and FSD operates perfectly fine during that time.

 

[spacecoin]

And as Tesla is getting so close, all the previous distractions are being proved wrong. LIDAR companies are going out of business because LIDAR doesn't solve the FSD problem.

Define close? They're not anywhere close to an MTBF that is required to remove supervision and take on liability. I think Tesla are 3-4 orders of magnitude from removing supervision. The main reason to add multiple sensing modalities and hd-maps is to add nines/reliability. They are not required "to drive".

People here seem to actually think that Waymo couldnt drive without maps and Lidar. In reality they can't drive with the required safety metrics required for unsupervised operation in a safety critical context.

It's easy to remove maps and extra sensing modalities if camera-only should be deems safe enough at some point in the future. Right now, it's a lot safer to be in a car with Lidar. It's not very expensive, so why risk it?

 

[ewoodrick]

Define close? They're not anywhere close to an MTBF that is required to remove supervision and take on liability. I think Tesla are 3-4 orders of magnitude from removing supervision. The main reason to add multiple sensing modalities and hd-maps is to add nines/reliability. They are not required "to drive".

People here seem to actually think that Waymo couldnt drive without maps and Lidar. In reality they can't drive with the required safety metrics required for unsupervised operation in a safety critical context.

It's easy to remove maps and extra sensing modalities if camera-only should be deems safe enough at some point in the future. Right now, it's a lot safer to be in a car with Lidar. It's not very expensive, so why risk it?

Have you been driving in the US? How many hours have experienced FSD or are you watching YouTube videos and coming to a conclusion?

 

[ewoodrick]

That said, more sources of information can always be used to enhance safety. I still think Lidar and radar would be valuable additions to the sensor suite, to compensate for Pure Vision's shortcomings in rain/fog/dirt or low light. If Robotaxi without Lidar could be 99.9999% reliable, perhaps Robotaxi with Lidar could be 99.999999% reliable.

The easiest way to explain this could you drive better with additional data sources?

Tesla got rid of radar because it added very little value and combining the two data sources is problematic.
When two data sources disagree, how do you determine the correct one? Does it even make a difference?

If you look at some of the internal visualizations that Tesla lets out periodically, detection and classification doesn't seem to be a problem. It is able to construct a scene with dramatically more detail than a human ever does. Think of it, can you tell me the number of cars surrounding you at any time?

The problem, AFAIK, is not in the detection and classification, it's in the what the hell do I do with it.

Is great precision required to drive? Are you able to determine the distance to the car in front of you in inches? Do you need to?

 

[OxBrew]

Have you been driving in the US? How many hours have experienced FSD or are you watching YouTube videos and coming to a conclusion?

I have about 100 miles of free trial 12.3.x, and I had to slam in the brakes about 5x to not actually crash. One time, cars coming down a hill from the right at 45mph, car started to cross, right into their path. Would have been a major T-Bone 1 second later.

Another time it almost crashed into a tree at a fork in the road.

5 critical failures in 100 miles. It needs to go 600k to a million miles. How close is it?

Spokane, Wa.

 

[enemji]

I still think Lidar and radar would be valuable additions to the sensor suite, to compensate for Pure Vision's shortcomings in rain/fog/dirt or low light. If Robotaxi without Lidar could be 99.9999% reliable, perhaps Robotaxi with Lidar could be 99.999999% reliable.

The only place this would be useful is where there are no road rules - ie no stop signs, no traffic lights, no lane markings, no school zones, no speed limits. Basically off-road situations.

 

[enemji]

The problem, AFAIK, is not in the detection and classification, it's in the what the hell do I do with it.

I have been saying that since 2022 when I got my first M3AWD. Too much data, and not enough intelligence to make decisions based on that data. We have come a long way in 24 months though. Drove 40 miles yesterday. No takeovers. Point A to B, driven smoothly under chill mode.