Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Were HW3-gen NNs running passively since July?

Mardak

Member
Oct 13, 2018
665
1,318
USA
I just see no evidence of path planning/driving policy running full time when AP is not engaged.
A lot of greentheonly youtube videos with Autopilot overlay are done without Autopilot active. The overlays include driveable space, obstacles, lane lines, stop lines; and additionally for the main camera there's the thick orange line for the path planner output.

Similar to how the NN running on HW2/.5 outputs stop signs but isn't visualized, the NN is also predicting a path but not displayed normally.

Although for regular Tesla owners, maybe one could trigger an Autopilot safety mechanism that relies on a predicted path… I guess quickly approaching a vehicle on a curve could highlight a vehicle not directly ahead as red?
 
  • Helpful
Reactions: strangecosmos2

strangecosmos2

Koopa Troopa
Nov 24, 2019
177
120
New Donk City
So, is the consensus that the Autopilot path planner runs passively in the background when Autopilot isn’t engaged? If so, that all but confirms shadow mode, as Elon originally described it, does indeed exist in some form.

We have already established that the computer vision system runs passively in the background, so in that sense the computer vision system runs “in shadow mode”. Furthermore, I’m fairly certain green has established that the computer vision system triggers snapshots and uploads when it’s running passively in the background and Autopilot is disengaged. So, it seems fair to call that shadow mode.
 
  • Disagree
Reactions: croman

tyson

Member
Nov 13, 2016
479
386
IA
Well if that's true then the path planner in the video you posted from Green intended to drive through the firetruck..... Path does not indicate full driving policy. Driving policy should have shown that it would stop, hopefully.

I think there is consensus that SOME things are running in the background but not a full "if I were driving I would do this" sort of thing. Now, on AP2/2.5 you could ask green directly as he does have root on those boxes.
 

strangecosmos2

Koopa Troopa
Nov 24, 2019
177
120
New Donk City
I suspect the orange ribbon only represents part of the Autopilot planner and the part that stops for/follows a vehicle ahead (e.g. while lane-keeping on the highway) is not visualized in green’s videos. I’ve asked green to weigh in on this thread; hopefully he can shed more light on the matter.
 
  • Disagree
Reactions: croman

EVNow

Well-Known Member
Sep 5, 2009
9,471
28,167
Seattle, WA
So, is the consensus that the Autopilot path planner runs passively in the background when Autopilot isn’t engaged? If so, that all but confirms shadow mode, as Elon originally described it, does indeed exist in some form.

We have already established that the computer vision system runs passively in the background, so in that sense the computer vision system runs “in shadow mode”. Furthermore, I’m fairly certain green has established that the computer vision system triggers snapshots and uploads when it’s running passively in the background and Autopilot is disengaged. So, it seems fair to call that shadow mode.
Here is something I can say definitively.

All the required systems for Tesla to determine the objects being visualized - AND - systems needed to determine whether AP can be engaged or not are running in the background all the time. That is why we can see visualization - as well as AP ready (wheel) symbol come and go.

Green has shown code which shows various kinds of triggers are possible.

I guess shadow mode doesn't mean the same to everyone and sceptics will always argue there is no shadow mode (using a version fo the definition that accepts only 100% shadow mode). But clearly some kind of shadow mode is running and Tesla is using it to collect training data.
 

arghx7

Member
Aug 6, 2019
487
495
Michigan
Tesla probably has a much smaller fleet, such as employee vehicles and test vehicles, running a “full” shadow mode With prototype code. Just because it’s not running on thousands of cars on standard release software doesn’t mean It doesn’t exist.
 

strangecosmos2

Koopa Troopa
Nov 24, 2019
177
120
New Donk City
Tesla probably has a much smaller fleet, such as employee vehicles and test vehicles, running a “full” shadow mode With prototype code. Just because it’s not running on thousands of cars on standard release software doesn’t mean It doesn’t exist.

There is a small alpha tester fleet (including Elon's car) where the cars will semi-autonomously drive on residential streets, dense city streets, highways, and parking lots. That's way beyond shadow mode.

Depending how you define "shadow mode", some form of it is probably running on regular customer cars with HW2+.
 
  • Disagree
Reactions: croman

verygreen

Curious member
Jan 16, 2017
2,914
11,298
TN
12:50 in this video.

Isn't the "orange ribbon" in green's videos the path planner/path predictor running passively in the background?
it seems to be result of "stay in lane" manager plan. Even when executing lane changes or swerving to stay away form trucks it always shows paath into the middle of current lane even while the car performs some other action.
 

About Us

Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.

Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


SUPPORT TMC
Top