Current HW2 Autopilot using 2 of 8 cameras * Testing Inside *

[Bladerskb]

Except this point is likely wrong. It seems that the main and the wide are active.

not at all. only the main camera is active.

Autosteer erratic/flashing on/off, road markings in the middle of the road started showing up in instrument cluster (but no right hand side markings shown)

The reason why the lane marking is even showing up is because he didn't cover the main camera all the way through.
comparing that image with whats on tesla autopilot site. Because he slanted the tape, the main camera would still be able to see the bottom left.
This is why only the left lane markings show up.

 

[larmor]

Curious as to how many of the camera on the new Model S are actually being used at the moment so I decided to do some testing today.

I started by first covered in the side fender cameras. This has zero effect on Autopilot and it worked as if they were not covered.

I then covered the side B pillar cameras. Again, zero effect on Autopilot.

I then covered the windshield drivers side camera. Again no effect and no error warning.

Next I covered the windshield passenger side camera and finally started to see an effect. The Autopilot steering wheel icon kept appearing and disappearing. I was able to engage Autosteer but it was pretty erratic.

I finally covered all cameras and Autosteer would not work at all, as expected. The odd thing was that the visual lanes would still pop in and out of view on the display but the AP steering wheel icon never appeared.

Looks great-- how about Mission Impossible mode: take a picture from a view of the camera(s) and with a selfie stick position it in front of the camera to simulate the road condition.

 

[lunitiks]

Or just tap into the coax wire going to the PX2

 

[Mobius484]

Thanks for you approach to this evolution of technology. I have been spending as much time as possible getting to know my AP 2 system, and this is helping me understand greatly... I appreciate your sleuthing!

 

[DJ 240V]

would love to get your input Do you feel safe? [ Calling all HW2 EAP pilots.]

 

[DJ 240V]

So I covered it more:
View attachment 216707
Now autosteer was completely unavailable.

When all your cameras are fully covered does TACC work ? Tryin to figure out if TACC relys on cameras at all. There is another thread debating about if radar or cameras are used in stop and go traffic.

 

[lunitiks]

They're both at work in AP1, since the camera must verify that the radar is actually "seeing" a car (and not just something harmless or out of the way) before the TACC is allowed to brake/accelerate. "Cruise not available" happens in my car all the time because of (A) snow/ice buildup in front of radar, or (B) ice/condensation in front of the camera.

No reason to believe AP2 is any different at this stage

 

[chillaban]

Yeah it's really not feasible to have anywhere near an acceptable TACC experience without cameras. Radar gives relatively crude information about everything that's in front of you. But when you take a curve with a car next to you, it's the camera that tells you the other car isn't in your lane and you assume the other car is following the lane lines. Same for driving toward a metal guardrail that the radar can pick up. On the flip side, when you are approaching stopped traffic around a curve, it is the camera that helps predict that a car isn't in your collision path according to radar, but is actually in your lane so you should slow down.

Without camera assistance, TACC must either assume the worst and always slow down for every non threatening object that you may collide with if going straight, or assume the best and assume everything isn't a threat until it's nearly in front of you. Neither will work all that well.

All half-decent ACC systems on the market are sensor fusion of a camera and a LIDAR/RADAR system. The camera being the more important piece.

 

[Fingerz]

Yeah it's really not feasible to have anywhere near an acceptable TACC experience without cameras.

Not so on my Mercedes (2013 G63). The Distronic Plus radar cruise control does not rely on any camera to a pretty darn good job of maintaining my set distance from the car/bike in-front of me, even in shallow bends. In the Tesla, it works exactly the same way (even using the same physical controls and probably the same Bosch radar components and controller).

 

[chillaban]

Not so on my Mercedes (2013 G63). The Distronic Plus radar cruise control does not rely on any camera to a pretty darn good job of maintaining my set distance from the car/bike in-front of me, even in shallow bends. In the Tesla, it works exactly the same way (even using the same physical controls and probably the same Bosch radar components and controller).

I don't recall the non-camera Distronic Plus (which by the way has 4 edge mounted K-band radar sensors up front too for directional front radar sensing at short ranges, while Tesla only uses the single long range radar with a much more limited field of view) handling high speed differential (e.g. Approaching extremely slow / stopped cars) well.

Mercedes eventually ended up adding a single camera and then 2 years ago dual cameras... the G class seemed stuck on older tech for one reason or another.

Back in 2008, all the HELLA supplied ACC systems were camera-less. They used to work alright for maintaining highway speeds but they certainly would not be as capable as AP1 or even AP2.

 

[bjornb]

When all your cameras are fully covered does TACC work ? Tryin to figure out if TACC relys on cameras at all. There is another thread debating about if radar or cameras are used in stop and go traffic.

I believe it was available, but I didnt try it. After reading the other posts above I'm starting to doubt myself whether I remember things correctly (I was focusing on Autosteer when testing). Its easy to do another test though..

Yeah it's really not feasible to have anywhere near an acceptable TACC experience without cameras. Radar gives relatively crude information about everything that's in front of you. But when you take a curve with a car next to you, it's the camera that tells you the other car isn't in your lane and you assume the other car is following the lane lines. Same for driving toward a metal guardrail that the radar can pick up. On the flip side, when you are approaching stopped traffic around a curve, it is the camera that helps predict that a car isn't in your collision path according to radar, but is actually in your lane so you should slow down.

I disagree, my previous car (an Audi) had a single radar only ACC (16 deg FOV and 4 beams) and it worked fine with some limitiations, but was still very useful on the road. It did also handle moderate curves, by using steering wheel angle to calculate a 'virtual' lane to decide wether a car was in the current or adjacent lane:


I also had the limitation that it only worked on moving objects using doppler shift to calculate speed (it did not respond to stationary cars, or other 'scenery'):

 

[chillaban]

I believe it was available, but I didnt try it. After reading the other posts above I'm starting to doubt myself whether I remember things correctly (I was focusing on Autosteer when testing). Its easy to do another test though..



I disagree, my previous car (an Audi) had a single radar only ACC (16 deg FOV and 4 beams) and it worked fine with some limitiations, but was still very useful on the road. It did also handle moderate curves, by using steering wheel angle to calculate a 'virtual' lane to decide wether a car was in the current or adjacent lane:
View attachment 217522

I also had the limitation that it only worked on moving objects using doppler shift to calculate speed (it did not respond to stationary cars, or other 'scenery'):
View attachment 217523

Audi switched to using a camera in their more recent ACC systems too…

I believe it was available, but I didnt try it. After reading the other posts above I'm starting to doubt myself whether I remember things correctly (I was focusing on Autosteer when testing). Its easy to do another test though..



I disagree, my previous car (an Audi) had a single radar only ACC (16 deg FOV and 4 beams) and it worked fine with some limitiations, but was still very useful on the road. It did also handle moderate curves, by using steering wheel angle to calculate a 'virtual' lane to decide wether a car was in the current or adjacent lane:
View attachment 217522

I also had the limitation that it only worked on moving objects using doppler shift to calculate speed (it did not respond to stationary cars, or other 'scenery'):
View attachment 217523

There was a period of time when automakers tried this approach before cameras were cost-effective, but nobody likes this approach anymore. Audi also switched to camera fusion in all of their newer vehicles, starting with that 2011 A8 platform that bubbled up to the 2013 A6 and the newer A3/A4's.

Also, note that because Audi strictly markets their ACC solution as a driver assist, it generally errs towards assuming that it does not need to brake.

It's definitely possible to make a more fancy ACC system based off advanced processing and kinematic models based off a radar system with no computer vision, but as a former automotive engineer I find it extremely implausible that once they have a camera, they would invest as heavily into this functionality without a camera. And it seems like AP2 HW is much more prone to false braking for overhead signs and incorrect handling of vehicles bordering on your lane, so….

 

[bjornb]

Audi switched to using a camera in their more recent ACC systems too…


There was a period of time when automakers tried this approach before cameras were cost-effective, but nobody likes this approach anymore. Audi also switched to camera fusion in all of their newer vehicles, starting with that 2011 A8 platform that bubbled up to the 2013 A6 and the newer A3/A4's.

Also, note that because Audi strictly markets their ACC solution as a driver assist, it generally errs towards assuming that it does not need to brake.

It's definitely possible to make a more fancy ACC system based off advanced processing and kinematic models based off a radar system with no computer vision, but as a former automotive engineer I find it extremely implausible that once they have a camera, they would invest as heavily into this functionality without a camera. And it seems like AP2 HW is much more prone to false braking for overhead signs and incorrect handling of vehicles bordering on your lane, so….

I know they introduced cameras in later models, and I agree that was the sensible way forward, but the radar only based ACC was still useful on the road as a driver assist system (I believe the Euro VW Golf and Passat still use radar only ACC).

 

[chillaban]

I know they introduced cameras in later models, and I agree that was the sensible way forward, but the radar only based ACC was still useful on the road as a driver assist system (I believe the Euro VW Golf and Passat still use radar only ACC).

Sure, agreed. Hopefully Tesla is looking way past a useful driver assist system right now. It would be interesting to experiment with how AP2 HW handles having the cameras totally blinded. But given my experience with AP1 and with 3 camera-based Audi's, they do not seem happy about it and barely allow ACC to activate if the camera is obscured.

 

[lunitiks]

This should be easy to test. Can't believe you get TACC with full blindness

 

[malcolm]

Can't believe you get TACC with full blindness

Because the car isn't blind; you can't block a radar sensor or the ultrasonics with just tape.

(I guess you could tape metal plates over them)

 

[lunitiks]

I meant camera blindness. I don't think TACC works without camera feedback. But I'd honestly love to be wrong

 

[malcolm]

In fog or dust storms we need a design of TACC that can work without cameras.

Sadly Mobileye seem to be fixated with optical solutions.

 

[lunitiks]

Very good point. Actually kind of a mic dropper. Hadn't thought of that (stupid me). TACC certainly must work in conditions where the camera can't make out the car ahead of you - that's kind of one of the selling points of the radar: It sees stuff that you and your camera can't.

What is needed, are some more tests that can verify/disprove that TACC (yes, TACC, we have already tested Autosteer) does not work when the cams are totally blacked out, by masking tape or some other light blocking material.

 

[bjornb]

I did test this today, covered up the cameras again:


TACC is still available and works fine (lanes are missing of course):


The direction of the car in front is not always shown correctly (I assume the camera is used for that):


I also looked for how it acted in curves and towards cars in adjacent lanes, and it handled it fine. The car did not lock on to or follow the speed of cars in adjacent lanes in curves, which means Tesla also does some kind of calculation similar to the Audi radar only ACC approach I described in my post above.
When you think about it (as @lunitiks also says above), it is a logical and sensible approach to make the system as robust as possible so it can still work when one or more sensors does not give useful data at the moment.