Were HW3-gen NNs running passively since July?

[Mardak]

I just see no evidence of path planning/driving policy running full time when AP is not engaged.

A lot of greentheonly youtube videos with Autopilot overlay are done without Autopilot active. The overlays include driveable space, obstacles, lane lines, stop lines; and additionally for the main camera there's the thick orange line for the path planner output.

Similar to how the NN running on HW2/.5 outputs stop signs but isn't visualized, the NN is also predicting a path but not displayed normally.

Although for regular Tesla owners, maybe one could trigger an Autopilot safety mechanism that relies on a predicted path… I guess quickly approaching a vehicle on a curve could highlight a vehicle not directly ahead as red?

 

[strangecosmos2]

So, is the consensus that the Autopilot path planner runs passively in the background when Autopilot isn’t engaged? If so, that all but confirms shadow mode, as Elon originally described it, does indeed exist in some form.

We have already established that the computer vision system runs passively in the background, so in that sense the computer vision system runs “in shadow mode”. Furthermore, I’m fairly certain green has established that the computer vision system triggers snapshots and uploads when it’s running passively in the background and Autopilot is disengaged. So, it seems fair to call that shadow mode.

 

[tyson]

Well if that's true then the path planner in the video you posted from Green intended to drive through the firetruck..... Path does not indicate full driving policy. Driving policy should have shown that it would stop, hopefully.

I think there is consensus that SOME things are running in the background but not a full "if I were driving I would do this" sort of thing. Now, on AP2/2.5 you could ask green directly as he does have root on those boxes.

 

[strangecosmos2]

I suspect the orange ribbon only represents part of the Autopilot planner and the part that stops for/follows a vehicle ahead (e.g. while lane-keeping on the highway) is not visualized in green’s videos. I’ve asked green to weigh in on this thread; hopefully he can shed more light on the matter.

 

[EVNow]

So, is the consensus that the Autopilot path planner runs passively in the background when Autopilot isn’t engaged? If so, that all but confirms shadow mode, as Elon originally described it, does indeed exist in some form.

We have already established that the computer vision system runs passively in the background, so in that sense the computer vision system runs “in shadow mode”. Furthermore, I’m fairly certain green has established that the computer vision system triggers snapshots and uploads when it’s running passively in the background and Autopilot is disengaged. So, it seems fair to call that shadow mode.

Here is something I can say definitively.

All the required systems for Tesla to determine the objects being visualized - AND - systems needed to determine whether AP can be engaged or not are running in the background all the time. That is why we can see visualization - as well as AP ready (wheel) symbol come and go.

Green has shown code which shows various kinds of triggers are possible.

I guess shadow mode doesn't mean the same to everyone and sceptics will always argue there is no shadow mode (using a version fo the definition that accepts only 100% shadow mode). But clearly some kind of shadow mode is running and Tesla is using it to collect training data.

 

[strangecosmos2]

Just noticed one instance where Karpathy used the term "shadow mode" on Autonomy Day:

 

[arghx7]

Tesla probably has a much smaller fleet, such as employee vehicles and test vehicles, running a “full” shadow mode With prototype code. Just because it’s not running on thousands of cars on standard release software doesn’t mean It doesn’t exist.

 

[strangecosmos2]

Tesla probably has a much smaller fleet, such as employee vehicles and test vehicles, running a “full” shadow mode With prototype code. Just because it’s not running on thousands of cars on standard release software doesn’t mean It doesn’t exist.

There is a small alpha tester fleet (including Elon's car) where the cars will semi-autonomously drive on residential streets, dense city streets, highways, and parking lots. That's way beyond shadow mode.

Depending how you define "shadow mode", some form of it is probably running on regular customer cars with HW2+.

 

[strangecosmos2]

This aged poorly:

Simple. Tesla misleads. There is no real shadow mode. Anyone who believes it should also keep leaving cookies and milk on the 24th.

(No offense, croman.)

 

[croman]

This aged poorly:



(No offense, croman.)

I'll pray for you.

 

[verygreen]

12:50 in this video.

Isn't the "orange ribbon" in green's videos the path planner/path predictor running passively in the background?

it seems to be result of "stay in lane" manager plan. Even when executing lane changes or swerving to stay away form trucks it always shows paath into the middle of current lane even while the car performs some other action.