Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Tesla begins HD map development (collecting short camera videos) as i predicted

This site may earn commission on affiliate links.
Okay cool, that's what I was thinking too, really hoping for some cool EAP feature to kick off Model 3 launch.

last big AP update end of April... are you referring to the update at end of April, where they added low speed AEB... that's defiantly an important update, but most people won't see it as big update.. maybe you referring to the update at the end or march that added full speed auto steer, auto lane change, and auto summon?

You're right, sorry I meant end of march.


however there is one important piece to this, where it has to detect cars in the side lane to make sure it doesn't hit them obviously. but I imagine EAP can continue to do this fine with ultrasonic, I see no reason to not keep using ultrasonic for the cars on its side...

However it will need to detect cars in that lane that are approaching, in the side lane, that are a long ways back, like out of range of the ultrasonic sensors...

I imagine it would have to use the 2 rear facing cameras to do this.... So I would guess at somepoint soon there will be an OTA update, where someone realizes that those back cameras are now being used. (right now only 2 front cameras are being used). And then an update some point after that which would enable the automatic lane changing. Does this logic make sense to you?

Well, judging from the reviews of auto lane changes. Its been a hit and miss. works roughly 50% of the time. Sometimes it shows there's an object which isn't there, or it doesn't see the car that's there. Plus its limited to 8 meters distance, something with that high percent of false positives and false negatives can't be trusted. So that's why I believe it will be put out of its misery completely.

The rear view FOV is able to see cars on the side.

autopilot_cameras-side_rear--touch.png






though I suppose those GPS maps would be sufficient to know if a lane is an exit lane or is about to close... maybe?

Being accurate to acouple meters could put you in another lane.

11 mins 0 secs
 
You won't see and FSD features until after the public demo in Nov-Dec time frame Elon had given for the NY to LA demo.

They will more then likely release those features as safety features first as they won't need any or very little regulator approval to stop at a stoplight automatically if someone is about to blow through it. This will be part of the process for gaining full regulatory approval. Imagine sitting in front of the NHSTA and showing them video of the car stopping people from going through a red light or a stop sign and causing a major accident. Now take that times 1000 videos captured over a few months. Regulators will be tripping over themselves to get that approved.

That sounds DANGEROUS as you don't know the drivers intent. This is why FSD cannot work as a safety backup feature that activates because driver intent isn't known and its certainly not what tesla will be doing. Collison avoidance is already doing what any FSD safety feature could do.

I'm confident that they will release the full FSD software and field it as driver assistance when they can get around 1 in 10/100 miles disengagement and they won't need any regulatory approval.
 
  • Like
Reactions: croman
Thanks Bladerskb! Also really appreciate some of the sources you've put down.

Personally really excited they're sending video data back now. I've been tracking network usage of the car over wifi and it just hasn't been enough for video data in the past and I've always scratched my head thinking "what are they doing to train the networks?" (Being optimistic that they were, but knowing realistically they probably weren't as much as I thought). Now I know!

I don't have 17.17.4 yet but pretty interested to see if the video clips are uploaded at night when the car is connected to wifi or just over LTE. Does anyone know the resolution of the cameras? I imagine they're actually reasonably low so uploading a ~30s clip over LTE is probably pretty inexpensive. But... they may store them until the evening anyways. We'll see!

I'm also interested if with the data clip sharing if they have are capturing video from all cameras now (or more than 2, at least). I expect only 2 are still used for actual AP behavior but I wonder if specifically the video clip capture will capture from more cameras. There is no reason they can't be annotating different perspectives while the cameras aren't being actively used.

Hoping to get the update tonight... I'm still stuck on 17.11.3.

Super interesting.
 
What seems more likely is the short video clips are going to be used to help improve Tesla Vision's object recognition. When it doesn't properly detect objects, Tesla will have the video clips to review - and possibly use to re-test new software updates to correct any mistakes in the object recognition.

It also seems unlikely that "HD mapping" means building a 3D model of every road, locating the specific position of every traffic sign and stop light. It's more likely "HD mapping" will help determine the specific locations of each lane, intersection, and on/off ramps - in combination with the GPS location and navigation database, the higher resolution mapping information will help the software know exactly where it is - on the specific road and in a specific lane. And for that HD mapping, tracking the GPS location of the car as it is moving may be the data Tesla needs to build an HD map of how each road/intersection/highway is laid out.

For FSD to work properly, it can't rely 100% on its database, because road conditions change unexpectedly - and the software has to be able to handle those situations. Road have temporary construction detours, changes in speed limits, shifting lanes, covered/new street signs.

Every time a driver disengages AutoSteer, it would be very helpful for Tesla to get a detailed data dump (including a short video from the cameras) to help determine why the driver didn't trust AutoSteer. And as Tesla makes progress in recognizing traffic signs/signals, the video could also be useful, especially when the onboard software encounters what it believes to be a sign or signal, and can't figure out how to interpret it.

Tesla's been capturing driving data to help support AP development - evidently their legal staff must have determined that capturing the video may need an explicit approval before Tesla can accept the video.

Of course, we are all guessing on this - unless Tesla decides to provide more information on what they are doing and how they are going to use the video they are collecting... So we could all be wrong on this...
 
What seems more likely is the short video clips are going to be used to help improve Tesla Vision's object recognition. When it doesn't properly detect objects, Tesla will have the video clips to review - and possibly use to re-test new software updates to correct any mistakes in the object recognition.

I agree with high confidence that the clips will be used for improving object recognition. However... ;) (read on)

It also seems unlikely that "HD mapping" means building a 3D model of every road, locating the specific position of every traffic sign and stop light. It's more likely "HD mapping" will help determine the specific locations of each lane, intersection, and on/off ramps - in combination with the GPS location and navigation database, the higher resolution mapping information will help the software know exactly where it is - on the specific road and in a specific lane. And for that HD mapping, tracking the GPS location of the car as it is moving may be the data Tesla needs to build an HD map of how each road/intersection/highway is laid out.

I disagree with this. GPS isn't accurate enough, and OP has posted various sources showing as such. GPS is accurate enough to put you within a couple lanes, but you really need secondary (or more) sources to help place you exactly in the right lane.

Beyond specific location (and assuming you get that), OP is also right from the paper linked that identifying the proper light is a critical requirement. Its not something that can easily be determined "on the fly" and it could be dangerous to do so since getting that wrong could result in going when you shouldn't.

For FSD to work properly, it can't rely 100% on its database, because road conditions change unexpectedly - and the software has to be able to handle those situations. Road have temporary construction detours, changes in speed limits, shifting lanes, covered/new street signs.

My expectation is that initial FSD features (not FSD itself perhaps) will rely on exact knowledge through multiple sources and if those sources aren't available, it won't allow FSD or require takeover.

I don't think point A to point B navigation under FSD will be purely use the most efficient path. I think it will create paths based on the subset of roads that it has sufficient data on and I expect that this may be limited to major roads for awhile. I think local neighborhoods may be different though (low risk to not have HD mapping on those since speed limits are low and traffic is low assuming you have high enough confidence in object detection and radar/ultrasonics). But for major streets I think it'll stick to specific ones.

Over time this will probably lax...

As you said: everyone is just guessing. I respect your opinion though. I'm basing my own opinion on my background in software and the papers and talks I've been trying to keep up with but for all I know you're doing the same with a differing opinion or from a more experienced position. Cheers.
 
You won't see and FSD features until after the public demo in Nov-Dec time frame Elon had given for the NY to LA demo.

What makes you say that?? Why would they not release some features earlier, then demonstrate more advanced fetures later on? and then release the advanced features when they are ready?

Why would Elon say there was going to be FSD features in April - July, if that was not the plan.. sure Elon is overly optimistic, but he would not make something up.

They will more then likely release those features as safety features first as they won't need any or very little regulator approval to stop at a stoplight automatically if someone is about to blow through it. This will be part of the process for gaining full regulatory approval. Imagine sitting in front of the NHSTA and showing them video of the car stopping people from going through a red light or a stop sign and causing a major accident. Now take that times 1000 videos captured over a few months. Regulators will be tripping over themselves to get that approved.

Soo Tesla could to that.. but users would be pissed! and I don't think they would collect many videos that way.

If Tesla updates autopilot to stop at stop signs, they would also need no regulatory approval, so that argument does hold any weight.
 
You're right, sorry I meant end of march.




Well, judging from the reviews of auto lane changes. Its been a hit and miss. works roughly 50% of the time. Sometimes it shows there's an object which isn't there, or it doesn't see the car that's there. Plus its limited to 8 meters distance, something with that high percent of false positives and false negatives can't be trusted. So that's why I believe it will be put out of its misery completely.

The rear view FOV is able to see cars on the side.

autopilot_cameras-side_rear--touch.png








Being accurate to acouple meters could put you in another lane.

11 mins 0 secs


but I mean, they will still use the camera for lane centering like they already do. they will not use the map for positioning, only for determining if the lane next to me is a valid lane.


What do you mean lane change works 50% of the time? what happens the other 50?

but yea so I am agreeing with you Tesla will add those rear facing cameras for lane changing
 
That sounds DANGEROUS as you don't know the drivers intent. This is why FSD cannot work as a safety backup feature that activates because driver intent isn't known and its certainly not what tesla will be doing. Collison avoidance is already doing what any FSD safety feature could do.

I'm confident that they will release the full FSD software and field it as driver assistance when they can get around 1 in 10/100 miles disengagement and they won't need any regulatory approval.
agreed
 
It is not the message, it is the way it is presented that makes people oppose him....

That is not an uncommon excuse, but IMO for fact gathering we really should learn better. Facts can come from many sources. The style, the presentation or even the post count of the source should not be used to judge. It has been misleading us too often. Many important data points have been dismissed too easily, too readily, too hastily because of such excuses.

I think we would be wise to listen to everyone with great care. Filter out all emotional kneejerks and displeasures as noise. Then analyze the potential data beneath - what fits and what not.
 
but I mean, they will still use the camera for lane centering like they already do.

True.

they will not use the map for positioning, only for determining if the lane next to me is a valid lane.

True aswell but the only problem with using gps map for valid lane is that if you're trying to take an exit and your lane detection network is saying your in the middle lane and your gps map is saying don't worry your in the right lane, just proceed. what do you do? discard the gps map data and do a lane change anyway?


What do you mean lane change works 50% of the time? what happens the other 50?

I mean I have seen a lot of complaints of it not working (refusing to change..etc). The 50% is a rough estimate and isn't an accurate predictor of its success rate
 
Last edited:
That sounds DANGEROUS as you don't know the drivers intent. This is why FSD cannot work as a safety backup feature that activates because driver intent isn't known and its certainly not what tesla will be doing. Collison avoidance is already doing what any FSD safety feature could do.

I'm confident that they will release the full FSD software and field it as driver assistance when they can get around 1 in 10/100 miles disengagement and they won't need any regulatory approval.

No one intends to run a red light or blow through a stop sign. No one intends to drift into another Lane where a car is occupying the Lane. I am not saying it's going to take over every time a driver does something different then the machine, but similar to AEB it can easily be overridden by turning the wheel, tapping the break or the accelerator.

I would dangerous not to implement this capability of you had it.
 
What makes you say that?? Why would they not release some features earlier, then demonstrate more advanced fetures later on? and then release the advanced features when they are ready?

Why would Elon say there was going to be FSD features in April - July, if that was not the plan.. sure Elon is overly optimistic, but he would not make something up.



Soo Tesla could to that.. but users would be pissed! and I don't think they would collect many videos that way.

If Tesla updates autopilot to stop at stop signs, they would also need no regulatory approval, so that argument does hold any weight.

Tesla doesn't need drivers to collect videos for them, they have access to your card feeds now, unless you want to tape over your cameras.

I agree no regulatory requirements to stop at a stop sign in autopilot. I was speaking of FSD proof and basis for approval.

Tesla doesn't have some huge test fleet, we are it. This why there is a shadow mode and why Tesla told us AP2 was running in shadow mode for everyone and learning. Same will happen for FSD. At the demo Elon might say, and this has been running in your cars in shadow mode since xxx date and will be activated in xxx weeks, usually 2 weeks and misses by a couple of months but it eventually rolls out.

Elon never said FSD I'm April - July time frame, I thought the same thing. After going back and really focusing on his yet and reply, he clearly says the that FSD features would start to differentiate from EAP in that time range. Not be completed and start to roll out. Look AEB, the validation and false positives will require a lot of data. The same data will be used for regulatory approval. I would have more faith if AEB was already out. One good thing is FSD doesn't require radar so it might not be held up by whatever is giving AEB false positives.
 
The OP may be as smart as he thinks he is, but he appears to not know GPS too well

"High-end users boost GPS accuracy with dual-frequency receivers and/or augmentation systems. These can enable real-time positioning within a few centimeters, and long-term measurements at the millimeter level."
GPS.gov: GPS Accuracy

There are many sources of GPS information but this one summarizes pretty well. With good ground mapping of altitude and a GPS received that is multi-frequency and fast enough quite reliable location accuracy within <10cm is quite common today. People in this thread, including the OP seem to be equating typical smartphone GPS with current state of the art. I will not suggest that use of GPS can eliminate need for sign-reading and other local data, nor that GPS will function perfectly without very clear reception of satellite signals.

When combined with receptors for GLONASS, the Russian GPS equivalent, more dependable high accuracy measurements are possible because of adding more satellite positions to the geometric calculations. That sometimes helps also because the US maintains the right to intentionally degrade the GPS signals, although they've not done that very much. The biggest weakness of these systems is not accuracy. It is that they are military systems used civilly by the generosity of the military masters.

In practice today GPS can and does quite dramatically reduce the requirement for other data in open areas, and even in areas that have intermittent obstructions such as mountains, forests and cities. Even in those areas combining GPS with inertial navigation can maintain those levels of precision and accuracy. BTW, deductive ( often incorrectly called 'dead' reckoning cannot substitute because it assumes maintaining an established course, almost never a good assumption on a road.

Because of the complexity of these issues multiple techniques can be effectively employed. IMHO most people who says they know the exact answer to a given question usually don't actually understand the question very well. People who do understand the question usually are a bit modest because they usually understand that new information can change established assumptions.

In short, video clips probably help but they certainly are not anything like a complete answer. They obviously contribute information, but are transitory in their value because of the dynamic nature of surface navigation.

As usual I point out I am not an expert. I have used GPS since 1987, and kept studying the advances since then. So I do think I'm pretty knowledagble, for a layman.
 
At least for the Toyota, Lexus and Tesla navigation systems I've used, they appear to use a combination of GPS data and measurement of the car's motion and direction to estimate the car's location.

Sometimes the Tesla navigation system will lose sync with the GPS data and position the car at the wrong spot on the map. When the car moves, the map is also moving the location of the car (even if it's on the wrong street), and eventually the car regains the GPS sync, and then moves the car to the correct location.

Tesla should be able to get pretty accurate GPS location data, which is made more accurate when the car is in motion - coupled with the motion data, Tesla should be able to produce pretty accurate measurements of the lanes, intersections, and on/off ramps, even at a higher resolution than the actual GPS data.

One of my concerns about FSD is how much work is going to be required with the onboard navigation software to get it to work well enough to replace human navigation. The current navigation software has multiple problems - it's using very out-of-date mapping data (which can be 2+ years old), plus the software sometimes comes up with some creative routing on side streets, that human drivers wouldn't take (some of these are based on traffic conditions, and sometimes they are mistakes).

Tesla hasn't made any comments about replacing the combination of Navigon software, out-of-date navigation map data, Tesla's user interface/trip planner and the Google maps/satellite data displayed on the console.

I'm guessing on this... Rather than investing in a new navigation software system in the cars, which will always have problems with out-of-date navigation data, Tesla could use the HD mapping data base they are creating and move the routing to a cloud server. That's essentially what's already being done for locating destinations - the destinations appear to be searched using Google and not the on board navigation database (which is why even very recent road/business changes can be searched for destinations).

If Tesla did this - the cloud server would be used to produce the route - the turn list would be downloaded into the car's software, and the navigation package/EAP/FSD would use that routing.
 
The OP may be as smart as he thinks he is, but he appears to not know GPS too well

"High-end users boost GPS accuracy with dual-frequency receivers and/or augmentation systems. These can enable real-time positioning within a few centimeters, and long-term measurements at the millimeter level."
GPS.gov: GPS Accuracy

There are many sources of GPS information but this one summarizes pretty well. With good ground mapping of altitude and a GPS received that is multi-frequency and fast enough quite reliable location accuracy within <10cm is quite common today. People in this thread, including the OP seem to be equating typical smartphone GPS with current state of the art. I will not suggest that use of GPS can eliminate need for sign-reading and other local data, nor that GPS will function perfectly without very clear reception of satellite signals.

When combined with receptors for GLONASS, the Russian GPS equivalent, more dependable high accuracy measurements are possible because of adding more satellite positions to the geometric calculations. That sometimes helps also because the US maintains the right to intentionally degrade the GPS signals, although they've not done that very much. The biggest weakness of these systems is not accuracy. It is that they are military systems used civilly by the generosity of the military masters.

In practice today GPS can and does quite dramatically reduce the requirement for other data in open areas, and even in areas that have intermittent obstructions such as mountains, forests and cities. Even in those areas combining GPS with inertial navigation can maintain those levels of precision and accuracy. BTW, deductive ( often incorrectly called 'dead' reckoning cannot substitute because it assumes maintaining an established course, almost never a good assumption on a road.

Because of the complexity of these issues multiple techniques can be effectively employed. IMHO most people who says they know the exact answer to a given question usually don't actually understand the question very well. People who do understand the question usually are a bit modest because they usually understand that new information can change established assumptions.

In short, video clips probably help but they certainly are not anything like a complete answer. They obviously contribute information, but are transitory in their value because of the dynamic nature of surface navigation.

As usual I point out I am not an expert. I have used GPS since 1987, and kept studying the advances since then. So I do think I'm pretty knowledagble, for a layman.

I respectfully disagree. gps cannot put you within cm unless everyone else will be using. you would need costly dgps units with ground based reference station to get <10 cm.

Differential Global Positioning System (DGPS) is an enhancement to Global Positioning System that provides improved location accuracy, from the 15-meter nominal GPS accuracy to about 10 cm in case of the best implementations.

Differential GPS - Wikipedia

The gps used by automakers and tesla gives you accuracy of acouple meters even when combined with imu of the car.
Again we are not talking about what possible. We are talking about facts. There are cameras that can see 1,000 miles away, but they are not in cars. We know what gps are used in cars and they are accurate to acouple meters.

Here is a demonstration below. Again this isn't opinion this are facts from Nvidia. Unless you are telling me you are much knowledgeable than Nvidia engineers?

GPS + IMU measurements in cars.
11 mins 0 secs
[/
 
I respectfully disagree. gps cannot put you within cm unless everyone else will be using. you would need costly dgps units with ground based reference station to get <10 cm.



Differential GPS - Wikipedia

The gps used by automakers and tesla gives you accuracy of acouple meters even when combined with imu of the car.
Again we are not talking about what possible. We are talking about facts. There are cameras that can see 1,000 miles away, but they are not in cars. We know what gps are used in cars and they are accurate to acouple meters.

Here is a demonstration below. Again this isn't opinion this are facts from Nvidia. Unless you are telling me you are much knowledgeable than Nvidia engineers?

GPS + IMU measurements in cars.
11 mins 0 secs
[/
1. DGPS is not the same as dual-frequency. Both are more expensive than the cheap ones now in common use principally because accuracy is less important in cars than in some other applications. both are much cheaper than, say, lidar.

2. If NIVIDIA engineers are alleging that "a couple of meters" is the best GPS can do then, yes, I know better than that. I have no idea what their motivations or biases might be.

People, even very accomplished ones, often view things in terms of their established biases. It is that establishment bias that allowed SpaceX to develop reusable rocket stages. It is that that allowed Tesla to use 'laptop cells' to power a long distance BEV.
There is a pretty well developed conventional wisdom about precisely how level 5 autonomy can be achieved. Since it has to yet been done it is safe to assume that some techniques that ease that development will be those which will reduce ad hoc information tracking to the minimum. Nobody knows exactly how that will happen.

Bluntly, statements of absolute certainty about this, or any other, subject are quite likely to make me react badly. When somebody says "I told you so..." that somebody usually has very selective memory. Selective memory is a poor basis for decision making.
 
  • Like
Reactions: Duke-U
True.



True aswell but the only problem with using gps map for valid lane is that if you're trying to take an exit and your lane detection network is saying your in the middle lane and your gps map is saying don't worry your in the right lane, just proceed. what do you do? discard the gps map data and do a lane change anyway?




I mean I have seen a lot of complaints of it not working (refusing to change..etc). The 50% is a rough estimate and isn't an accurate predictor of its success rate

Lane changing is working 100 pct for me. It just isn't as smooth as the operation of AP1 - the transition is more jerky. But as for accuracy, speed and consistency it's just as good if not better than my AP1 car.
 
  • Like
Reactions: 1 person
True.



True aswell but the only problem with using gps map for valid lane is that if you're trying to take an exit and your lane detection network is saying your in the middle lane and your gps map is saying don't worry your in the right lane, just proceed. what do you do? discard the gps map data and do a lane change anyway?




I mean I have seen a lot of complaints of it not working (refusing to change..etc). The 50% is a rough estimate and isn't an accurate predictor of its success rate

Ah, I see what you are saying. valid point.

So then this leads me to believe Tesla cannot enable auto lane change, or on/off ramp, until they have HD map of certain highways.
So that means Tesla cannot enable these features until get enough HW2 cars to drive around our freeways enough.
Right, I'm sure it doesn't take long to map freeways, especially in areas where there are lots of Teslas.

Do you think its possible both for EAP features, and for FSD features, that Tesla will geolock to only certain areas at first, and graudally enable more areas as the maps are built?

Elon never said FSD I'm April - July time frame, I thought the same thing. After going back and really focusing on his yet and reply, he clearly says the that FSD features would start to differentiate from EAP in that time range. Not be completed and start to roll out. Look AEB, the validation and false positives will require a lot of data. The same data will be used for regulatory approval. I would have more faith if AEB was already out. One good thing is FSD doesn't require radar so it might not be held up by whatever is giving AEB false positives.

Elon said there will be a noticeable feature, that differentiates people who bought fully self driving opposed to just enhanced autopilot. and that feature would come around late april to late july.

Tesla to transition from ‘Enhanced Autopilot’ to ‘Fully Self-Driving’ as soon as ‘3 to 6 months’, says Elon Musk


Why do you think FSD will not use radar?
 
Elon said there will be a noticeable feature, that differentiates people who bought fully self driving opposed to just enhanced autopilot. and that feature would come around late april to late july.

Tesla to transition from ‘Enhanced Autopilot’ to ‘Fully Self-Driving’ as soon as ‘3 to 6 months’, says Elon Musk


Why do you think FSD will not use radar?

Apparently you are not the only one to misread an Elon musk tweet. I thought the same thing when I first read the tweet, but if you re-read the original tweet and the response and dont try to read into what you want to happen, he is not saying what you think he is saying. What Elon said was that in 3-6 mo, EAP and FSD would "start to differentiate" from each other, not be completed and released to the public. The keyword there being "start", not finish.

Why no radar? Because Elon said, less then 2 weeks ago in the TED talk, that they would use GPS and Vision only for FSD. These are what we call actual facts, directly from one of very few people who actually knows. Could he be lying? I guess. Could they change their minds, sure.