Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

I got my new Autopilot defeat device

This site may earn commission on affiliate links.
funny, but I just drove 2 hours with AP running 30.5.1 and hanging my arm/wrist through the steering wheel hole

no nags and was not staring straight on the road

think they are solely relaying on sensing the arm weight on the steering wheel
 
Last edited by a moderator:
funny, but I just drove 2 hours with AP running 30.5.1 and hanging my arm/wrist through the steering wheel hole
no nags and was not staring straight on the road
think they are solely relaying on sensing the harm weight on the steering wheel
Tests that I have performed with my MY (no FSD) had no evidence of the interior camera used for AP day or night.

I may run a test using my M3. It has FSD, which certainly uses the camera. I'm wondering if the camera is still used if I disable FSDb and use AP only. Just don't understand why some cars use the camera with basic AP and some do not.
 
funny, but I just drove 2 hours with AP running 30.5.1 and hanging my arm/wrist through the steering wheel hole
no nags and was not staring straight on the road
think they are solely relaying on sensing the harm weight on the steering wheel
IME Tesla uses a combination of inputs to determine attention - steering wheel torque OR use of any of the controls OR gaze. Gaze alone seems to delay/spread out the naga but not eliminate them so it seems they use it to augment but not replace the other methods.
 
  • Like
Reactions: timberlights
I doubt the cabin camera is infrared. This is an urban legend in the making. It can be viewed with camera preview and it obviously is not infrared. Also, look at the IR pics below. Even if the cabin camera was a thermal imaging IR camera, it certainly won't detect eyeball orientation with or without glasses.
1000025968.jpg
1000025966.jpg
 
I couldn't find any definitive information about the camera being IR, but this picture from a Reddit post shows an IR view in the dark:

View attachment 1001157

Hopefully it works reliably. The IR cameras that Ford used for Blue Cruise, for example, were 100% faultless in my experience. I had no false positives, and it was nearly impossible to "fool" the system into thinking you were actually looking at the road. Granted I only logged a few dozen hours with BlueCruise, but it was still a pretty impressive implementation. Ford uses two cameras in front of the steering wheel, however, and not a wide angle centrally placed camera like Tesla does.
That's not IR, it's low light maybe aided with an LED light.
 
  • Like
Reactions: flstcgranucci
The Tesla patent package for determining driver attentiveness cites facial recognition and head positioning:

https://t.co/i2shU6nko6

What parts of this patent document were actually implemented is a mystery to me. But if eye monitoring is a major component, it may explain why dark sunglasses may reduce nag frequency. "What big eyes you have." , said Little Red Full Self Driving Hood.
 
I doubt the cabin camera is infrared. This is an urban legend in the making. It can be viewed with camera preview and it obviously is not infrared. Also, look at the IR pics below. Even if the cabin camera was a thermal imaging IR camera, it certainly won't detect eyeball orientation with or without glasses.View attachment 1003109View attachment 1003110
I misspoke, the in cabin camera does have an IR illuminator. Else the night time picture in my garage below would show nothing. But I still am hesitant to believe eyeball direction is used to determine attentiveness. Also some have said an IR camera can see through sunglasses. These IR cabin cameras can not see through regular or sunglasses as the IR light bounces off the reflective surface.
1000025981.jpg


Security cameras are utilizing infrared night vision, which relies on infrared light to function. This infrared (IR) light is completely invisible to the naked eye. Think of your current TV remote, which has a red LED at the front of the device and sends an infrared signal to your television. You cannot see this beam transmitting to your TV as it is in the infrared focal plane. The same can be said about IR illuminators and IR lights. These devices are typically mounted on the camera itself and shine a spotlight of infrared light, allowing these devices to capture images/video and view in total darkness.
 
Last edited:
TLDR: Thermal-IR and typical surveillance type near-IR are two quite different applications. The Tesla cabin camera almost certainly uses inexpensive near-IR, which at minimum allows the AI driver-attention algorithms to work at night without distracting the driver. Eye tracking also uses near-IR but it's not clear to me that the Tesla cabin camera will be very good at true precise eye tracking.

As mentioned by others above, there is a range of infra-red wavelengths used for different Imaging applications.

Most IR-sensitive cameras, including home security cameras and more pertinently the Tesla cabin camera (at least the current ones*) have sensitivity in what is usually called "near infrared", essentially adjacent but just outside the human-visible deep-red end of the color spectrum.

In most cases, these use active illumination to achieve so-called night vision capability, these days using cheap near-IR LEDs to "light up" the subject for the camera without visibly lighting it up as seen by our eyes.
(In this respect, it's undisturbing and unobtrusive illumination, but it's active and not very "secret" in a military or law enforcement application, because it's quite easy to see the illumination source with an inexpensive camera.)​
So, as the famously repurposed cabin camera has become progressively more important for ensuring appropriate driver attention behavior in the L2 Autopilot/FSD modes, Tesla elected simply to add an IR illuminator function so that the driver monitoring could be reasonably effective at night.

This near-IR capability is almost free and is not the same as IR thermal imaging for picking up heat signatures in total darkness. Thermal imaging of people, animals and typical vehicles must provide sensitivity at much longer IR wavelengths, i.e. far deeper into the infrared spectrum because the photon radiation energy of not-very-hot things is much lower.
Yes, thermal imaging does have limited applications in the near-IR, but objects glowing significantly in this range are nearly "red-hot". Sorry, but most Tesla occupants, whatever they may think of themselves or their dates, are not Hot enough to register at night without the help of the active near-IR illuminator. ;)

Back to the application we are talking about here: to understand why this is free or very low cost, it's important to note that this range can easily be picked up within the Red color hannel of a typical inexpensive RGB digital camera sensor. This is an advantage for the actively-illuminated "night vision" surveillance-camera usage as mentioned. However, it's a disadvantage for accurate photography and videography, because materials that are particularly reflective in the near-infrared (including some clothing) will unnaturally brighten up the image and appear too bright red or otherwise off-color in an RGB image, or simply too light altogether in a monochrome image.
This actually happened in some early digital cameras, including expensive Leica cameras (but also Nikon and others), and it was somewhat embarrassing that the problem had to be remedied by supplying IR-cut filters to attach over the front of the lenses. These were supplied for free or at significant discount as a kind of "recall remedy" for the early adopters who paid a lot of money but sometimes got rather strange results. Follow-on designs paid more attention to fancy multi-coated filters over the sensors - but now people sometimes hack the caneras to remove these again for astronomical, special effects, surveillance or just nerdy hobbyist reasons.​
* I have no idea if the Tesla cabin cameras, the pre-IR illuminated versions, had IR-cut filters or not. I don't think it was a very significant issue either way, so probably whatever was cheapest at the time.​

Regarding eye tracking: I have no special expertise in this, but most eye tracking systems also do use Illumination in the near-IR spectrum. Notice that if you look at video of people and animals walking around in front of your home night-vision cameras, you will definitely see varying degrees of glowing eyes, due to the reflection and refraction of the eyeball over the retina.

I'm not sure, but I suspect that the above-mirror position of the cabin camera and IR LED probably doesn't correspond to the optimal position for a purposefully designed eye tracking setup. There's no doubt that the AI analysis software can be trained to estimate where you're looking, more or less the same way you can infer where someone is looking whenever you can see his/her face. But I'm not sure that the "real" geometrical IR-return eye tracking method can be accomplished by Tesla's setup, as effectively as it may be in a purpose-built setup as with Ford Blue Cruise or similar systems.

Finally, while on this topic I'll note one more thing: I wonder why ADAS designers, including Tesla, don't add near-IR diode emitters to enhance the exterior night vision of the vehicle. It would help in a number of scenarios, depending on the power level of the emitters, but even relatively low-power and unfocused IR LEDs would help a lot up to moderate distances. I suspect it would also help in wet and foggy conditions due to lower scatter, and I think that a version of IR always-on high beams could improve the car's forward distance vision when visible high beams can't be used.

Sorry this is kind of rambling and not exactly on the topic of the OP joke, but I saw a lot of discussion about IR, thermal imaging and eye tracking. Hope everyone stays safe while using autopilot!