We can read how it works in Tesla's own words (bolding mine except for the numbers)
GENERATING GROUND TRUTH FOR MACHINE LEARNING FROM TIME SERIES ELEMENTS - Tesla, Inc.
"At
405, deep learning analysis of the sensor data is initiated. In some embodiments, the deep learning analysis is performed on the sensor data optionally pre-processed at
403. In various embodiments, the deep learning analysis is performed using a neural network such as a convolutional neural network (CNN). In various embodiments, the machine learning model is trained offline using the process of FIG. 2 and deployed onto the vehicle for performing inference on the sensor data.
For example, the model may be trained to identify road lane lines, obstacles, pedestrians, moving vehicles, parked vehicles, drivable space, etc., as appropriate. In some embodiments, multiple trajectories for a lane line are identified. For example, several potential trajectories for a lane line are detected and each trajectory has a corresponding probability of occurring. In some embodiments, the lane line predicted is the lane line with the highest probability of occurring and/or the highest associated confidence value. In some embodiments, a predicted lane line from deep learning analysis requires exceeding a minimum confidence threshold value. In various embodiments, the neural network includes multiple layers including one or more intermediate layers. In various embodiments, the sensor data and/or the results of deep learning analysis are retained and transmitted at
411 for the automatic generation of training data."
"In various embodiments, the
deep learning analysis is used to predict additional features. The predicted features may be used to assist autonomous driving. For example, a detected vehicle can be assigned to a lane or road. As another example, a detected vehicle can be determined to be in a blind spot, to be a vehicle that should be yielded to, to be a vehicle in the left adjacent lane, to be a vehicle in the right adjacent lane, or to have another appropriate attribute. Similarly,
the deep learning analysis can identify traffic lights, drivable space, pedestrians, obstacles,
or other appropriate features for driving."
"By training the machine learning model using a ground truth determined using image and related data of a time series that includes elements taken at the locations of labels A, B, and C of FIG. 5, three-dimensional trajectories of lane lines
601 and
611 are predicted with a high degree of accuracy even portions of the lane lines in the distance, such as portions
621. Although image data
600 and image data
500 of FIG. 5 are related, the prediction of trajectories does not require image data
600 to be included in the training data.
By training on sufficient training data, lane lines can be predicted even for newly encountered scenarios. In various embodiments, the predicted three-dimensional trajectories of lane lines
601 and
611 are used to maintain the position of the vehicle within the detected lane lines and/or to autonomously navigate the vehicle along the detected lane of the prediction lane lines. By predicting the lane lines in three-dimensions, the performance, safely, and accuracy of the navigation is vastly improved."
"Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive."
