Current HW2 Autopilot using 2 of 8 cameras * Testing Inside *

[lunitiks]

What the heck! In an article today, Electrek has picked up this thread but claims that the cars are only using the "Main" (passenger side) camera and nothing else.

I wonder what source Electrek has on this, and how it explains the fact that OP completely covered it up and still got Autosteer functionality. You don't get Autosteer with only the radar - if that was the case than OPs covering up of all cameras would have allowed Autosteer! (Which it didn't.)

Perhaps Electrek isn't being precise in the use of the word "Main". Maybe Electrek means the *mid* (center) camera, which is wide angle (Tesla calls it FishEye).

But bjornb has just proved that this cannot be the only camera at work.

More testing please!

 

[bjornb]

I did some more testing after work today.
I found a stretch of local roads with moderate curves and good road markings, like this:

I used this stretch of road with different combinations driving back and forth, and moving the bits of tape around.

Drivers side narrow cam available, autosteer possible, but erratic.



I also did test with only center wide cam available (but forgot to take a picture), autosteer possible, but erratic.
One thing I noticed when driving with only narrow or wide cam available was that it did not draw the road curves correctly, the slight curve shown in the picture above was a straight road in the instrument cluster.


Passenger side main cam, autosteer works fine!

Road curves was also shown correctly in the instrument cluster.

All cameras covered, autosteer still available occationally(!), but very erratic...

At this point I partially invalidated my own tests above, since this demonstrated that I didnt place my bits of tape far enough down to cover the cameras completely (the cameras were still able to see the road right in front of the car)..

So I covered it more:

Now autosteer was completely unavailable.

At this point I am leaning towards that the passenger side main cam is the primary camera, and the two others (wide and narrow) only have a support function if any function at all at this point.

I will try to do some more testing tomorrow.

 

[lunitiks]

When I saw your initial pic, I thought to myself that maybe you should've placed more tape below, but I didn't think it mattered based on OPs tests so I didn't say anything.

When you're testing next time, try putting longer strips of tape in vertical direction, starting with the narrow (driver side) cam, then working your way towards the main (passenger side).

Then the same procedure again, but in opposite order.

When covering only the middle cam, make sure you cover it in such a way that the two side cams have visibility, but reducing the middle cams FOV as much as possible.

I think a couple of more tests will answer the community's questions.

Actually, if someone had the nuts to disassemble the glove box, I could provide this person with the necessary info on which connectors one could unplug to verify these tests (and more)

 

[Victor3]

Bjornb, I think your use of the word "main" leads to semantic confusion. Most people would instinctively think that main would mean the center camera, it being central to the triple stack. I know at first read I thought your reference to main meant middle camera. Just my 2c.

Who's on first?

 

[lunitiks]

Me.

"MAIN" is Teslas designation of the passenger side camera, being the one with the "average" FOV between the three.

Bjornb is just being accurate

 

[Victor3]

Me.

"MAIN" is Teslas designation of the passenger side camera, being the one with the "average" FOV between the three.

Bjornb is just being accurate

Source?

 

[Bladerskb]

What the heck! In an article today, Electrek has picked up this thread but claims that the cars are only using the "Main" (passenger side) camera and nothing else.

I wonder what source Electrek has on this, and how it explains the fact that OP completely covered it up and still got Autosteer functionality. You don't get Autosteer with only the radar - if that was the case than OPs covering up of all cameras would have allowed Autosteer! (Which it didn't.)

Perhaps Electrek isn't being precise in the use of the word "Main". Maybe Electrek means the *mid* (center) camera, which is wide angle (Tesla calls it FishEye).

But bjornb has just proved that this cannot be the only camera at work.

More testing please!

Can't say I didn't I tell you so. Every thing I have said on this forum since day one has been bull's-eye.

Elon on the conference said they were using the two rearward facing cameras and the two redundant forward camera.

On the Tesla autopilot site it lists the main camera and narrow camera as the redundant forward cameras.

Plus the main camera is the exact same one used in the AP1 mobileye setup.

Lastly knowing the Tesla only needs one camera to replicate AP1. I knew they only used the main camera

 

[lunitiks]

Yeah let's get this 100% verified @Bladerskb before you get your cake I'm still not ruling out that two cams are at work here, allthough I admit things are pointing in the direction that you're right.

Awaiting @bjornb tests tomorrow and we'll see

 

[lunitiks]

Plus the main camera is the exact same one used in the AP1 mobileye setup

If everything you've said on this forum from day 1 has been bulls-eye, could you please clarify this quote? I assume we're not supposed to take it literally?

 

[chillaban]

I wonder what the fleet must be learning from this....

 

[lunitiks]

Source?

Source? My brain is telling me this for a couple of reasons. In sum these reasons make me come to the conclusion of which camera is which. I might be wrong but I think I'm right:

1 - Wiring diagrams and connector references:

According to Teslas wiring diagrams and connector references, both the "FishEye" and "Narrow" cameras are connected to the the left (driver side) part of the triple camera enclosure on the same, double Rosenberger connector. The "Main" camera is connected to the right (passenger side) part of the triple camera enclosure.

2 - Appearance:

Anyone can tell that the three AP2-cameras *look* different. In my view (again, I might be wrong on this), the driver side camera appears to have the longest focal length, because it seems to be the one sticking furthest out - it's the "longest" camera of the three. The middle cam seems to have the shortest focal length, because it looks like the "shortest" camera. If the above is correct, then the passenger side cam is then of course the medium focal length; the "MAIN" camera.

Triple Cam (1)

• lunitiks
• Feb 28, 2017

Speaking of appearances, the middle camera looks a lot like a typical fisheye lens. Between the driver and passenger side cameras, I think the driver side cam looks like it's designed for more magnification than the passenger side one.

Triple Cam (2)

• lunitiks
• Feb 28, 2017

3 - A sensible placement of the FishEye camera:

Even if I'm not correct in my eye measurments of the camera lengths, or if the different lenghts don't matter, I think it's fair to assume that the camera with the biggest FOV - which according to Tesla is 120 degrees - should be placed in the middle of this triple camera rig. Why? Because on both sides of the rig you have "walls" that effectively block any light coming in. Why put the wide angle lens right next to one of these "walls", if you're interested in a widest possible view? Doesn't make sense.

If I'm wrong, I'm happy to be corrected.

 

[bjornb]

New day, more testing

I covered the cameras all the way today:


No autosteer at all, no road marking lines in instrument cluster.


No autosteer.


No autosteer.


Still no autosteer.


Autosteer erratic/flashing on/off, road markings in the middle of the road started showing up in instrument cluster (but no right hand side markings shown)

I didnt have any more time before work to do the same testing from the other side, so I just did this, and drove to work:

Autosteer working fine!

So I am still leaning towards that the passenger side main cam is the primary camera, and the two others (wide and narrow) only have a support function if any function at all at this point.

 

[Duke-U]

Anybody noticed the "Learning.." notice in the Driver Controls, Settings, Vehicle tab?? What would that mean? Does the notice go away when you cover up all camera's? I will try when I get the chance.

 

[MorrisonHiker]

Anybody noticed the "Learning.." notice in the Driver Controls, Settings, Vehicle tab?? What would that mean?

View attachment 216866

If you turn off Smart Preconditioning, does it go away?

 

[JohnnyG]

New day, more testing
.... <removed pics> .....
So I am still leaning towards that the passenger side main cam is the primary camera, and the two others (wide and narrow) only have a support function if any function at all at this point.

Here's a quote from the Tesla Website, supporting your findings:

Wide:
120 degree fisheye lens captures traffic lights, obstacles cutting into the path of travel and objects at close range. Particularly useful in urban, low speed maneuvering.

Main:
Covers a broad spectrum of use cases.

Narrow:
Provides a focused, long-range view of distant features. Useful in high-speed operation.

Sounds like the Wide can support Autosteer, but it's primary purpose is traffic signals / cross-path obstacles / maneuvering / etc.

 

[imispgh]

[/QUOTE]

 

[Matheus28]

At least back in 17.3.2, the car had files referring to a neural network called "main_narrow". I assume that it means it uses both. Your mileage may vary.

 

[lunitiks]

Thanks @bjornb for doing this. I think these experiments are very important in understanding what's going on. As of now, I think it's safe to say that:
• Passenger side (Main) cam is absolutely 100% certainly active
• Middle (Fisheye) cam is probably active as a fallback/redundant/secondary cam
• Driver side (Narrow) cam is probably not active, since covering it up had no effect just like the B-pillar cams, side repeater-cams and BUC.

(When I say "active", I mean that the cameras video stream is being processed by the AP2.0-ECU resulting in command output to the power steering, iBooster brake system and drive inverter.)

 

[Bladerskb]

If everything you've said on this forum from day 1 has been bulls-eye, could you please clarify this quote? I assume we're not supposed to take it literally?

I have said things that most people today come to take as facts.
But before I was killed for it.

EAP HW2 - my experience thus far... [DJ Harry] (only one camera active)

Enhanced Autopilot 2 may never happen before Full Self Driving is ready! (my accurate predicted timeline)

But I have been right for every one of it. whether its the delay or ap currently using only one camera, or the fact that mapping is required for meaningful self lane-change and off/on-ramp. Anything other than that would be a glorified ap1. The fact that fleet learning is a myth and the fact that shadow mode has taken a life of its own but is actually only a steering validation tool and nothing more.

 

[Max*]

EAP HW2 - my experience thus far... [DJ Harry] (only one camera active)

Except this point is likely wrong. It seems that the main and the wide are active.