How can Smart Summon data be used to train a neural net?

[Daniel in SD]

So I've seen the claim that Smart Summon data is being used to train some sort of neural net. I obviously have no idea how any of this is supposed to work. Wouldn't data from humans driving through parking lots be used to train a neural net to drive through parking lots? Or are people saying they take success/no success data from Smart Summon? Is there any evidence that Smart Summon is even using a neural net for path planning? There are an infinite number of wrong ways to drive through a parking lot so it seems like you'd need more information than just whether or not the car made it to its destination.

 

[Green Pete]

I don’t know for sure, but it’s my understanding that the neural net is only used for turning complex image data into categorical data. (Not friable area, dashed line, solid line, oncoming car, car going away, road signs etc). In that sense a summon is likely collecting a data of edge case data. Every time it stops abruptly I am guessing it’s making that video footage and sending it back to the cloud for labeling an analysis, and then in a subsequent release summon will have an improved neural net that is better at realizing that the patched crack in my parking lot should not justify the car freaking out. Pathing decisions likely only use the neural net to categorize potential things that block the road. It could be that some there is also some neural net stuff going in to deciding what might be a drivable area from a google maps view where a tree blocks much of the parking lot but I dunno.

all of the above is just a guess based on my limited understanding.

 

[heltok]

So I've seen the claim that Smart Summon data is being used to train some sort of neural net. I obviously have no idea how any of this is supposed to work. Wouldn't data from humans driving through parking lots be used to train a neural net to drive through parking lots? Or are people saying they take success/no success data from Smart Summon? Is there any evidence that Smart Summon is even using a neural net for path planning? There are an infinite number of wrong ways to drive through a parking lot so it seems like you'd need more information than just whether or not the car made it to its destination.

Collect situations where car was uncertain, have humans label the data.

Collect data where the car was uncertain but later figured out what was correct, use that to label data.

 

[Daniel in SD]

Collect situations where car was uncertain, have humans label the data.

Collect data where the car was uncertain but later figured out what was correct, use that to label data.

How does that require actually using Smart Summon though? Humans already traverse all the drivable space in parking lots every day. What's different about the Smart Summon image data?

 

[alsetym]

How does that require actually using Smart Summon though? Humans already traverse all the drivable space in parking lots every day. What's different about the Smart Summon image data?

I know with AP they have it running in the background all the time, potentially collecting data (and powering lane keep assist and such). They may be able to do something similar with smart summon, though all of this takes compute power so it may be limited. There are some differences though. It may not be able to classify fast enough at higher parking lot speeds for instance. Smart summon may also end up in somewhat unique situations because of the heuristics layer trying to make decisions that you don't come across often with a human driver.

 

[heltok]

How does that require actually using Smart Summon though? Humans already traverse all the drivable space in parking lots every day. What's different about the Smart Summon image data?

Very different to learn through observation and to learn through acting. Compare learning to play mario kart by watching a human play and by q learning.

 

[Daniel in SD]

Very different to learn through observation and to learn through acting. Compare learning to play mario kart by watching a human play and by q learning.

So what are the reward and punishment signals with Smart Summon?

 

[heltok]

So what are the reward and punishment signals with Smart Summon?

Backpropagation of the inference error.

 

[Daniel in SD]

Backpropagation of the inference error.

What is that?

 

[heltok]

What is that?

Backpropagation | Brilliant Math & Science Wiki

Basically what Tesla does is:
1. Train a neural network from labelled data using backprop of the inference error
2. Deploy NN, use NN uncertainty/error and intuiton to guide what data to gather
3. Label new data using human labelers and using self annotated data
4. Goto 2

The error at training is the reward. The difference between human driven data and machine driven data is that the failure cases might be different and need longer time horizons for them to become obvious. Just because a human and a machine would set ~same steering angle at Tzero doesn’t mean that the machine would not drift over time.