News

Musk Says Cars Will Detect Hand Gestures, Listen for Sirens

New details on Tesla Vision emerged over the weekend.

The camera-based vision system to power Tesla’s Full Self Driving feature is able to deliver interesting information to the driver. In a post to Twitter on Tuesday, Musk said Tesla Vision will have the ability to detect turn signals, hazards, police and ambulance lights, and hand gestures.

Musk was replying to a video from an FSD beta tester Eli Burton that showed the system detecting brake lights. Musk also said the cars will eventually listen for sirens and alarms to react to emergency vehicles.

Tesla’s camera-based system is an approach that differs from others developing self-driving systems using radar. Earlier this year the company announced the transition to Tesla Vision. Starting last May, Model 3 and Model Y vehicles built for the North American market stopped being produced with radar, instead relying on camera vision and neural net processing for the Autopilot and FSD.

Consumer Reports has reinstated the 2021 Tesla Model 3 with pure vision Autopilot as a “Top Pick” after a top safety designation from the Insurance Institute for Highway Safety.

Matias

Active Member
Apr 2, 2014
3,347
3,681
Finland
Lets see, what Musk said 2019; “he promised that Teslas will be capable of self-driving by the end of this year and self-driving robo-taxis will be on the road in 2020. Also, in two years, the company will be making cars without steering wheels or pedals at all.”

Source
 

qdeathstar

Completely Serious
May 17, 2019
3,196
2,225
VB

Attachments

  • 1626133915089.gif
    1626133915089.gif
    410.7 KB · Views: 4
  • Like
Reactions: ytfsic

father_of_6

Member
Jan 24, 2021
202
181
Buffalo, NY
Hey, I take a lot of what Elon Musk says with a grain of salt, too... but I applaud him for trying. While he seems to overestimate what his companies can achieve in a given time, it *still* tends to result in positive outcomes. This overestimation is typical of overachievers.

Maybe FSD is a big failure, but:
  • Tesla's cars are great
  • The Super Charger network is ahead of everyone else
  • SpaceX is kicking butt
  • Many people seem to be happy with StarLink
I'm sure there are many more positives to add to that list.
 

Carl Raymond

Active Member
Oct 18, 2018
1,524
12,454
NSW, Australia
The cars will listen…

Will this use the existing in-car microphone, or will it require an additional external microphone fitted?

As a mere human, I am sometimes fooled by siren like sounds in the music being played. A clever system will be able to subtract the cars‘ audio from the signal.

Cue the usual concerns of privacy advocates “Tesla is eavesdropping on our conversations“. A few hoops to jump through there.
 

thewishmaster

Member
Jun 4, 2018
548
432
California
I don't know whether to consider it a good thing that they're working on incorporating it, or alarming that it seemingly wasn't considered until now...

Of course the updated “4D” perception stack with multiple-frame inputs is much better suited for hand gestures and flashing lights on emergency vehicles, but still kind of surprising how far they went along with something that probably just couldn’t handle this at all.

Also the mic input will likely require new hardware with exterior mics because from the Twitter chatter it didn’t seem like there was any plausible way of using existing hardware, including the interior mic (not connected to AP, not to mention all the other noise interference one might encounter inside a car).
 
  • Informative
Reactions: Carl Raymond

Thp3

Member
Aug 21, 2017
512
475
Michigan
Also the mic input will likely require new hardware with exterior mics because from the Twitter chatter it didn’t seem like there was any plausible way of using existing hardware, including the interior mic (not connected to AP, not to mention all the other noise interference one might encounter inside a car).


Not necessarily. Sirens are designed for people to hear the siren horn while inside the car. So an internal microphone could be capable of hearing a siren. No more hardware is required.
 
  • Like
Reactions: Carl Raymond

thewishmaster

Member
Jun 4, 2018
548
432
California
Not necessarily. Sirens are designed for people to hear the siren horn while inside the car. So an internal microphone could be capable of hearing a siren. No more hardware is required.
The AP hardware would need to be able to access the mic for that situation, but my understanding is it would need to be wired to AP to use it effectively (eg the way the cameras/radar are wired directly to the AP board). But I don’t know what kind of other voodoo they can cast in order to get mic input from presumably the MCU computer.
 
  • Like
Reactions: Carl Raymond

Two-rocks

Member
Jan 18, 2021
150
183
gone
I’m going to put a siren sound on the boom box. Easy to clear other Tesla’s using FSD/autopilot outta my way…

If another car’s driver waves going by will they call the phantom braking a safety feature?
 
  • Like
Reactions: qdeathstar

gearchruncher

Active Member
Sep 20, 2016
1,936
2,520
Seattle, WA
Why is everyone here assuming that this will work on current Tesla cars? They already had HW2, 2.5 and 3, this is probably HW4 knowing how Elon works.

Also, note that "detect hand gestures" does not mean "react to hand gestures."

From the FSD v9 videos, seems like they should be focusing on "detecting huge concrete columns" first.