Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Profound progress towards FSD

This site may earn commission on affiliate links.
This is so wrong. Go watch Karpathy's most recent cvpr video.

You are clueless, get your head out the tesla bubble and learn something for once.

Everything listed below is sensing/perception.
The rest is classical control algorithm as Andrej confirmed at autonomy day to Tasha keeney
b7ar1crpdjyx.png
 
  • Like
Reactions: diplomat33
The traffic control feature uses mapping, lol. Talk about clueless.

I assume you don't even have a Tesla with the traffic control feature.

No the classical control algorithm uses map checks to make control decisions. In-addition to using the NN perception output of the lights and its classification

The actual traffic light detection and classification is under perception using a NN model and the control decision and actuation is done using classical control algorithms.

Again how is it that you cant understand? How many times do i have to repeat myself?
 
  • Like
Reactions: diplomat33
Again how is it that you cant understand? How many times do i have to repeat myself?

I don't know who or what you're responding to, but I said that the traffic control feature does all the following: "perception, mapping, prediction and planning."

It seems that you disagreed and replied by saying that all the traffic control feature does is NN perception and if statements to stop, which is wrong since it does all of the above.

Perception: light color and light position

Mapping: knows if there's a traffic control coming up from SD maps, also localizes the car within the lanes to better understand if the car is in a turn lane or not

Prediction: predicts whether the light is relevant to your current lane (also, NNs are predictions in general)

Planning: when and where to stop / should stop or not based on red / yellow

Again, I don't think you understand enough or have experience with the traffic control feature to fully appreciate it. I'm going to assume you didn't even know that it uses maps, so that's a big oversight.
 
Last edited:
During today’s Q2 earnings call, Elon mentioned several times how the new 4D version of FSD is a profound improvement over the current 2.5D stack. He he a damn good salesman and has me convinced.

It will be interesting to see what Tesla can achieve in the next 6 - 12 months.

Check out the video on the Topic. I think its really good information on autopilot history and 2.5d to 4d. What do you guys think?

Tesla Autopilot ReWrite 4D! & 3D labeling & Competition
https://youtu.be/kLukc3AtO-8
 
Check out the video on the Topic. I think its really good information on autopilot history and 2.5d to 4d. What do you guys think?

Tesla Autopilot ReWrite 4D! & 3D labeling & Competition
https://youtu.be/kLukc3AtO-8

Three observations:

1) 4D is self-driving 101. Drawing 3D boxes around objects is a basic tool of perception. Literally everybody, Mobileye, Waymo, Cruise etc already have 4D camera vision. So 4D camera vision is not some incredible breakthrough. It's foundational to any camera vision because the car has to see like the real world. So while it is great news that Tesla is close to releasing the 4D rewrite, it just means that Tesla's vision can now more accurately locate objects in the real world. That's just the basics of perception and a long way from solved FSD.

2) The video implies that Mobileye is aiming to do camera-only FSD like Tesla. That is misleading. The video fails to mention that Mobileye is developing 2 independent FSD systems, one that is camera-only, yes, but another that is lidar and radar only. And Mobileye plans to combine the two systems together in their final FSD system that goes into consumer cars. So Mobileye is still planning to use lidar in the FSD that they put in consumer cars.

3) The video praises how good the Mobileye car is at maneuvering in complex situations and seems to give all the credit to the 4D vision. The video implies that this is evidence that Tesla will be able to do similar self-driving when they finish 4D. But 4D is only a prerequisite, it does not automatically guarantee that you can handle those situations. The 4D vision is only the perception part. Advanced planning and driving policy are the real reasons that the Mobileye car is able to self-drive like that. So while Tesla doing 4D is an important prerequisite for being able to duplicate that demo, it does not guarantee it. Tesla still needs to do the planning and driving policy for those driving scenarios.
 
Last edited:
I don't know who or what you're responding to, but I said that the traffic control feature does all the following: "perception, mapping, prediction and planning."

It seems that you disagreed and replied by saying that all the traffic control feature does is NN perception and if statements to stop, which is wrong since it does all of the above.
That's all it does.

Perception: light color and light position
Perception

Mapping: knows if there's a traffic control coming up from SD maps, also localizes the car within the lanes to better understand if the car is in a turn lane or not

Nope tesla doesn't do localization and all it does is a map check from the classical control algorithm whether there's an intersection, traffic control or stop sign. Its not a fully integrated mapping system.

Prediction: predicts whether the light is relevant to your current lane (also, NNs are predictions in general)

That's called Traffic light relevancy network, that's also a perception network. EyeQ4 does that.
I don't think you know what prediction is in the context of autonomous driving. You are still using word play and refuse to learn.

Planning: when and where to stop / should stop or not based on red / yellow

That's called stop line detection network and road markings network. All perception networks. As the picture I showed you earlier. EyeQ4 which is a vision/perception chip does all of this. The classical control algorithms uses all of this to make decision to stop. go, or slow down, etc.

There are no prediction or planning (decision making) neural network in play at the moment.

Get educated.
 
Last edited:
There are no prediction or planning (decision making) neural network in play at the moment.

Lol, right. It seems all you're doing is defining terms in your head and then disagreeing with anyone who defines them differently. I already identified those 4 characteristics of the traffic control feature. Perhaps your definition of "planning" is different than mine or another company's.
 
  • Funny
Reactions: diplomat33
Lol, right. It seems all you're doing is defining terms in your head and then disagreeing with anyone who defines them differently. I already identified those 4 characteristics of the traffic control feature. Perhaps your definition of "planning" is different than mine or another company's.

Sure. The guy who works in the autonomous driving industry is just making up his own definitions. :rolleyes:

tenor.gif
 
Three observations:

1) 4D is self-driving 101. Drawing 3D boxes around objects is a basic tool of perception. Literally everybody, Mobileye, Waymo, Cruise etc already have 4D camera vision. So 4D camera vision is not some incredible breakthrough. It's foundational to any camera vision because the car has to see like the real world. So while it is great news that Tesla is close to releasing the 4D rewrite, it just means that Tesla's vision can now more accurately locate objects in the real world. That's just the basics of perception and a long way from solved FSD.

2) The video implies that Mobileye is aiming to do camera-only FSD like Tesla. That is misleading. The video fails to mention that Mobileye is developing 2 independent FSD systems, one that is camera-only, yes, but another that is lidar and radar only. And Mobileye plans to combine the two systems together in their final FSD system that goes into consumer cars. So Mobileye is still planning to use lidar in the FSD that they put in consumer cars.

3) The video praises how good the Mobileye car is at maneuvering in complex situations and seems to give all the credit to the 4D vision. The video implies that this is evidence that Tesla will be able to do similar self-driving when they finish 4D. But 4D is only a prerequisite, it does not automatically guarantee that you can handle those situations. The 4D vision is only the perception part. Advanced planning and driving policy are the real reasons that the Mobileye car is able to self-drive like that. So while Tesla doing 4D is an important prerequisite for being able to duplicate that demo, it does not guarantee it. Tesla still needs to do the planning and driving policy for those driving scenarios.


Yeah, those are some good points.
 
  • Like
Reactions: diplomat33
1) 4D is self-driving 101. Drawing 3D boxes around objects is a basic tool of perception. Literally everybody, Mobileye, Waymo, Cruise etc already have 4D camera vision. So 4D camera vision is not some incredible breakthrough. It's foundational to any camera vision because the car has to see like the real world. So while it is great news that Tesla is close to releasing the 4D rewrite, it just means that Tesla's vision can now more accurately locate objects in the real world. That's just the basics of perception and a long way from solved FSD.

You just described 3D, which works by analysing a sequence of static pictures over time (maybe a slight oversimplification). One of the challenges in this approach is making sure "Object 1" in picture 2 is the same entity as "Object 1" from picture 1, even though Object 1 in picture 2 looks a bit different.

What they are calling "4D" will analyse a continuous video feed, so is a fundamentally different approach.
 
  • Love
Reactions: mikes_fsd
You just described 3D, which works by analysing a sequence of static pictures over time (maybe a slight oversimplification). One of the challenges in this approach is making sure "Object 1" in picture 2 is the same entity as "Object 1" from picture 1, even though Object 1 in picture 2 looks a bit different.

What they are calling "4D" will analyse a continuous video feed, so is a fundamentally different approach.

No I was describing 4D. Analyzing a sequence of static pictures over time is what Elon calls 2.5D which is what we have now. The video even explains that. Tesla used to just have 2D bounding boxes on objects. Yes, analyzing video is 4D. I was describing 4D. You analyze video and label objects with 3D boxes in time. Elon even mentioned that the 4D rewrite uses 3D video labeling. That's 3D bounding boxes from video.
 
  • Disagree
Reactions: J1mbo and mikes_fsd
Just found a new patent filing from Tesla for "Estimating Object Properties Using Visual Image Data".

It looks like it's a patent that details using an image to identify a distance of an object from the vehicle using a machine learning model.

The patent says Tesla collects a series of images and radar data for a certain time period (~30 seconds, can be adjusted based on vehicle speed or distance traveled), it identifies a vehicle from the data and it is tracked across the time series, and determines a ground truth with a high degree of accuracy because the object is tracked through time any ambiguities such as multiple objects with similar distances or occluded objects can be resolved. Then the data is converted to training images and annotated with the ground truth data such as distance, velocity, acceleration, direction, and other appropriate object properties.

It also details the ability to gather training data "using auxiliary sensor data" and can accurately predict the result of the auxiliary sensor without the need of the physical auxiliary sensor.

rgeVrwy.jpg


That explains the sightings of Tesla MFG test mules with lidar/radar auxiliary sensors strapped to a rig.

Tesla-Autopilot-sensor-rig.jpg


Some information on how triggers work, and which scenarios may trigger the autopilot computer to capture data and send it to Tesla for usage as training data:

Iptzhol.jpg
 
Last edited:
Tesla has probably been thinking about doing the rewrite for years, but they were working on / validating the foundational vision-based capabilities that would allow for more precise 3D object labeling using all the cameras.

A 4D-based rewrite would never work if the vision-based detection and tracking of 3D objects was not good enough.

I'm not convinced that Tesla has cracked the code for precise and stable vision-based object distance and size detection yet. Even today, we still have cars and trucks dancing around on our in-car visualizations.
 
  • Funny
Reactions: scottf200