Current HW2 Autopilot using 2 of 8 cameras * Testing Inside *

[larmor]

Wild guess: spacex to launch satellites for tesla's own internet service to their cars. This will allow rapid data transfer and cross checking of data when extra cameras are enabled. Cars process their own data, but clearly they need communication with the mothership and other cars in order to enable FSD. current LTE connection is not great. just sayin'...

 

[lunitiks]

secondly the camera they are using is the main camera not the wide.

You're obviously not reading what OP just did:

I then covered the windshield drivers side camera. Again no effect and no error warning

So it's not the Narrow camera.

Next I covered the windshield passenger side camera and finally started to see an effect. The Autopilot steering wheel icon kept appearing and disappearing. I was able to engage Autosteer but it was pretty erratic.

So it's not the Main camera either - at least not only the Main camera.

Which leaves:

I finally covered all cameras and Autosteer would not work at all, as expected.

The FishEye camera, which is in the middle, with the largest camera lens and the widest possible FOV from its location in the windshield camera house. The masking tape on both sides of the FishEye camera was most likely what caused the "erratic" behaviour. When he took out the FishEye camera, Autosteer was completely gone

 

[Max*]

When he took out the FishEye camera, Autosteer was completely gone

Now you're not reading what the OP did.

When he took out the fisheye, he covered ALL cameras, so obviously Autosteer was gone. This proves that without ANY camera input, Autosteer doesn't work (duh!), but it doesn't prove that without the fisheye Autosteer doesn't work (it likely wont, but a separate experiment is needed).

 

[lunitiks]

It proves that the FishEye camera is the only camera needed for Autosteer functionality as of right now!

What we dont know, is whether another camera, e.g. the "Main" one, is also in use (redundancy).

So I agree, more experiments are needed

 

[stopcrazypp]

Main-FishEye-Narrow

• lunitiks
• Feb 26, 2017

I like this diagram, but I was curious about the reference for which camera is which. Is it just guessing from the cutout angles (or lens details) or some other source?

 

[Max*]

I don't think we'll be able to tell how many cameras EAP actually uses at this time, but we might be able to gather some more useful info. OP can you run a few more experiments for us?

While keeping the side cameras blocked out (4 of them, the fenders and B-pillars), can you do the following:
1. Block out the rear camera, the wide camera (center), and the drivers side camera [leaving only the passenger]
2. Block out the rear camera, the wide camera (center), and the passenger camera [leaving only the driver]

I think this will tell us which camera is the main camera.



Another thing to try, is to change lanes only the front 3 cameras active. Does it allow you to change lanes? How good is it at detecting cars in the neighboring lanes? (if the answer is it sucks more than usual, we would then know they side cameras are used)

 

[lunitiks]

@stopcrazypp :

First, you can tell by the lens sizes (relative to each other).

Second, you can deduce that the logical thing to do is to put the wide FOV in center of this rig, because of the "walls" on the sides.

Third, I've got some sources, yeah Bought access to the wiring diagrams and connector references.

Fourth and final: Isn't it interesting how the AP1.0-Model X camera setup is? With the only ("main") camera on the passenger side, and blindness in center and driver's side? This was supposed to be a MobilEye triple cam setup, I think. But it all went away with the dispute over Teslas SW / in house plans. Every time I see an AP1-X, I'm reminded of this negative story....

 

[MP3Mike]

Another thing to try, is to change lanes only the front 3 cameras active. Does it allow you to change lanes? How good is it at detecting cars in the neighboring lanes?

The lane change feature isn't available in AP2 cars yet, so this test would have to wait.

 

[lunitiks]

Auto lane change has been around for a long time with AP1.0. Can't see why covering up side cameras would prevent the functionality (when it arrives, that is)

 

[Max*]

Auto lane change has been around for a long time with AP1.0. Can't see why covering up side cameras would prevent the functionality (when it arrives, that is)

I never said that it would prevent it, I said if it gets worse:

How good is it at detecting cars in the neighboring lanes? (if the answer is it sucks more than usual, we would then know they side cameras are used)

But as MP3Mike pointed out, there is no "usual" yet, as it hasn't been enabled in AP2 cars yet (which I didn't realize).



Auto lane change works well 2/3 of times I use it. AP1 sucks at detecting cars in the neighboring lanes (*cough* blind spot detection *cough*).

The other 1/3 times it waits too long, and I either need to turn off my turn signal and turn it back on, or am forced to change lanes myself #firstworldproblems

 

[MP3Mike]

Auto lane change has been around for a long time with AP1.0. Can't see why covering up side cameras would prevent the functionality (when it arrives, that is)

Because EAP is supposed to use them to make sure it is safe. Something AP1 can't do.

My guess is that the only reason the feature exists in AP1 is so you don't have to disable Autosteer, change lanes, and then re-enable Autosteer. But you have to make sure it is safe before you tell the car to change lanes.

 

[RayW]

I thought I saw in another post that AP2 cars were running in "ghost mode" last year to correct for pitch angle and crop of the cameras, and some cars might need to have cameras repositioned at a SC.

 

[lunitiks]

I thought I saw in another post that AP2 cars were running in "ghost mode" last year to correct for pitch angle and crop of the cameras, and some cars might need to have cameras repositioned at a SC.

You're mixing up three different stories here

Shadow mode: Word is that *all* AP2.0-cars have been running in shadow mode, and probably still is when Autopilot functionality is not engaged. Shadow mode is simulated trial-and-error for the vehicles on board computer, taking with them the "learning" to Teslas mothership servers, where Tesla in turn can train their Neural networks and later send their validated SW back to the fleet.

Pitch angle issue: Elon Musk has tweeted that some of Tesla's AP2.0-vehicles (unknown how many) have incorrectly, physically mounted forward cameras in that their pitch angle is bad. This means that the cameras are either looking too much upwards or too much downwards.

SC-adjustment or Image cropping: Elon Musk tweeted that instead of sending a bunch of cars to the service centres, and spending time and money aligning cameras all day long, Tesla would try to implement a software update that "dynamically crops" the image within the camera/computer. Based on what I know about imaging, that would mean to crop (cut) away some of the image - first some of the top or bottom (depending on whether the camera is pointing too much up or down), and then some of the image sides in order to keep the aspect ratio and have it centered correctly. This last part is me guessing.

 

[Mobius484]

Wild guess: spacex to launch satellites for tesla's own internet service to their cars. This will allow rapid data transfer and cross checking of data when extra cameras are enabled. Cars process their own data, but clearly they need communication with the mothership and other cars in order to enable FSD. current LTE connection is not great. just sayin'...

I too have wondered how the internet satellites they are making in seattle, and the tesla fleet will integrate, you would think it would be super valuable to them, figured they will send em up on the reused rockets ASAP and hook us all up with satellite access!

 

[bjornb]

I also did a test on my way to work today, covered the center camera like this:

(I know its not completely covered, but I didnt want to affect any of the other cameras)

Autosteer (on 17.7.2) still worked as before..

 

[lunitiks]

Excellent! This means that at least 2 cams are at work.

Makes sense: EAP was supposed to have 2 "redundant" forward cameras.

 

[JRMW]

I've never understood why Tesla doesn't pull us into the loop more.

We are paying to be Beta Testers.
Why not help us Beta Test?
Why make us jump through all of these hoops, when you can harness all of this energy, time, and expertise FOR FREE?

What helps AP learn faster? If we drive normally (and it's in shadow mode), if we drive using TACC, or if we let Autosteer drive?
Why can't we just touch a "send report" button every time Autosteer or TACC make a mistake? This would give the mother ship better data collection

Give us the data and tools we need to be the best beta testers ever.

TACC was initially pretty scary (for me), especially with the hard braking for no reason
Then it suddenly got way better. I assumed it was because of an update
But now it's worse again. Doesn't do the hard braking, but does do random soft and medium braking. Why? Who knows.

Same with Autosteer. At first it tried to take every right exit that exists. Then seemed to get better. Now it's back to taking the right exits. Why? Who knows. Therefore I only use Autosteer in Middle and Left lanes, which deprives the fleet of right lane learning.

Tesla should help us help it.

 

[Max*]

Autosteer (on 17.7.2) still worked as before..

Awesome, thanks for the test.

 

[krazineurons]

what about parallel park, does it not need the side cameras?

 

[MP3Mike]

what about parallel park, does it not need the side cameras?

Doubtful at this point. It is probably just like AP1 in that it only uses the ultrasonic sensors at this point.