Your beliefs are groundless. Green has explicitly debunked your interpretation of his own findings. You have no car or primary sources.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Were HW3-gen NNs running passively since July?
- Thread starter strangecosmos2
- Start date
Your beliefs are groundless. Green has explicitly debunked your interpretation of his own findings. You have no car or primary sources.
Well actually he's right. Here's what verygreen said about it. He has truly debunked it
"We'll start with the bitter truth. The "shadow driver that just sits there in the computer comparing notes and sending discrepancies and interesting events to Tesla" is a myth. I used to think people just misunderstood Elon, but now I believe Tesla lies about it on purpose 2/"
Twitter
Twitter
"...There is a relatively small chance any particular hard brake/disengagement event is going to generate a snapshot for upload but only if all the other stars align right."
green on Twitter
green on Twitter
It has been debunked but some people can't let it go.
Last edited:
Sure but my points still stand. I said he isn't God and isn't all knowing but he's the best source and he sees enough to suss out shadow mode, as Tesla sees it, cannot exist.
Actually blader pulled a lot of them but his Twitter feed contains other examples of debunking shadow mode. It's just not there and even hw3s second node being activated just indicates redundant take over capability not AB comparisons or dueling minds like some fantasize about.
there's no "background shadow NN" pretty much makes this thread asked and answered.
green on Twitter
green on Twitter
Karpathy talks about the real fleet learning mechanism during the autonomy day. Starts at about the 2:05:00 mark. Basically they ask the fleet for images or situations using triggers. This is something green verified.
Now, some things do run behind the scenes, again something green verified when he posted about the stop light/stop sign detection being present some months ago. I had this feature turned on myself for a while before Tesla disabled it. The specific flag was "GUI_stopAndGo". It wasn't as advanced as what was just released but it detected stop signs and lights along with their states.
This wasn't some full NN running in the background just a portion that wasn't exposed to the GUI or used by the path planner/driving policy. So yes, of course some things run in the background without the user knowing or seeing, but the biggest method for fleet learning is still the triggered data snips, usually just pictures from what karpathy talked about.
Now, some things do run behind the scenes, again something green verified when he posted about the stop light/stop sign detection being present some months ago. I had this feature turned on myself for a while before Tesla disabled it. The specific flag was "GUI_stopAndGo". It wasn't as advanced as what was just released but it detected stop signs and lights along with their states.
This wasn't some full NN running in the background just a portion that wasn't exposed to the GUI or used by the path planner/driving policy. So yes, of course some things run in the background without the user knowing or seeing, but the biggest method for fleet learning is still the triggered data snips, usually just pictures from what karpathy talked about.
Then for those of us not in the programming/NN scene, can someone differentiate the difference between people talking about how visualizations have been around for the last year and shadow mode? As a layperson, to me that sounds like a method of running a program in the background. Which sounds like shadow mode. Or is the debate that path decision making is not compared and sent back to Tesla for analysis?
sorry if this is a very elementary question.
green's tweets (which I already linked to earlier in this thread) are consistent with how I envision shadow mode. I don't know where people are getting the idea that shadow mode means a car records data of every second it's in motion and uploads it all to Tesla. When did Elon ever say that?
In my mind, shadow mode is just event-based triggers of small data snapshots (e.g. maybe 5-second video clips) where the event is a disagreement between the (passively running) Autopilot planner's path and the human driver's path.
This wouldn't necessarily be a separate neural network running in the background; it could just be the same neural network(s) that run when Autopilot is active running in the background.
Aurora describes a quite similar idea in their ICML 2019 talk on slides 14-17 from 13:15 to 16:30.
In my mind, shadow mode is just event-based triggers of small data snapshots (e.g. maybe 5-second video clips) where the event is a disagreement between the (passively running) Autopilot planner's path and the human driver's path.
This wouldn't necessarily be a separate neural network running in the background; it could just be the same neural network(s) that run when Autopilot is active running in the background.
Aurora describes a quite similar idea in their ICML 2019 talk on slides 14-17 from 13:15 to 16:30.
Last edited:
I think this is the sticking point. I cannot speak for the planner, but object detection does run when autopilot is not engaged. This I have seen. As to the rest there is no evidence for it and all indications from Tesla are that disengagements are used to correct the path planner/driving policy. This was also discussed during the autonomy day speech by karpathy.
Indeed. green's videos demonstrate this. So, we know for a fact that computer vision NNs run passively in the background. Whether you want to call this "shadow mode" is purely semantic and irrelevant to the OP of this thread.
The evidence for it is simply that Elon's mentioned it in the past and that Aurora has also described a similar approach, lending credibility to the idea that this is something Tesla would try. This isn't definitive evidence that it is being implemented, but there also isn't any evidence to the contrary, as far as I know.
Also, what is Tesla uploading all to the time? I. The last month about a gig of data has been uploaded on my WiFi network, not sure how much gets uploaded on cellular.
I imagined most of this data was somehow relevant data to help improve AP. Is that magical thinking?
I imagined most of this data was somehow relevant data to help improve AP. Is that magical thinking?
Not at all. Tesla uploads data from the fleet to train neural networks on computer vision, prediction, and planning:
Link? Also Elon has said A LOT of things that never ended up coming true. So maybe add a touch of salt. I trust what Karpathy says in relation to AP far more than Musk. Karpathy writes the code ...
And again, they do gather a LOT of data... I just see no evidence of path planning/driving policy running full time when AP is not engaged.
Most times you can see this yourself if you force a disengagement of AP and monitor the cars data usage. It will wait for WiFi to upload. I have seen it not send data after a disengagement though as well.
Most times you can see this yourself if you force a disengagement of AP and monitor the cars data usage. It will wait for WiFi to upload. I have seen it not send data after a disengagement though as well.
12:50 in this video.
Isn't the "orange ribbon" in green's videos the path planner/path predictor running passively in the background?
If Tesla uploads a data snapshot for some/most/all crashes or "crash-like events" (which it apparently tracks, per its Safety Reports), and if that snapshot includes the "orange ribbon" and other planning-related data, then this would allow Tesla to determine whether Autopilot would have averted a crash or "crash-like event".
For example, if a Tesla owner drives off a cliff, and the orange ribbon is not pointing off the cliff, that's a good indication the crash would have been avoided on Autopilot.
Determining when Autopilot would have caused a crash when the human did not crash is trickier. That's why I hypothesize the system takes note whenever a human driver does not follow the orange ribbon.
Elon didn't say that the fleet would upload 100% of the instances where Autopilot would have either prevented or caused a crash. He just said the fleet would be used to collect a statistically significant sample size of such instances.
I also don't know if Elon ever said shadow mode would be used to collect data to train neural networks, as opposed to simply serve as a form of safety validation. The training part might just be my own hypothesis, extrapolating from what Elon and others have said.
TL;DR: Shadow mode probably exists in some form.
For example, if a Tesla owner drives off a cliff, and the orange ribbon is not pointing off the cliff, that's a good indication the crash would have been avoided on Autopilot.
Determining when Autopilot would have caused a crash when the human did not crash is trickier. That's why I hypothesize the system takes note whenever a human driver does not follow the orange ribbon.
Elon didn't say that the fleet would upload 100% of the instances where Autopilot would have either prevented or caused a crash. He just said the fleet would be used to collect a statistically significant sample size of such instances.
I also don't know if Elon ever said shadow mode would be used to collect data to train neural networks, as opposed to simply serve as a form of safety validation. The training part might just be my own hypothesis, extrapolating from what Elon and others have said.
TL;DR: Shadow mode probably exists in some form.
Last edited:
No that's the path prediction NN model that the classical control cpu path planning system uses as one of its input to drive.
Its not the actual path planning or driving policy system. It doesn't make decisions, it doesn't say go around this object or avoid this object or don't hit this. It simply draws a line straight ahead through a path. The line literally goes through an object in-front of it. its not the path planning system. The entire path planning is a classical control algorithm. cpu code.
This is directly contradicted by this
"We'll start with the bitter truth. The "shadow driver that just sits there in the computer comparing notes and sending discrepancies and interesting events to Tesla" is a myth. I used to think people just misunderstood Elon, but now I believe Tesla lies about it on purpose 2/"
Twitter
Twitter
Last edited:
Similar threads
