Blog Musk Says Cars Will Detect Hand Gestures, Listen for Sirens

New details on Tesla Vision emerged over the weekend.

The camera-based vision system to power Tesla’s Full Self Driving feature is able to deliver interesting information to the driver. In a post to Twitter on Tuesday, Musk said Tesla Vision will have the ability to detect turn signals, hazards, police and ambulance lights, and hand gestures.

https://twitter.com/x/status/1414435086548733952

Musk was replying to a video from an FSD beta tester Eli Burton that showed the system detecting brake lights. Musk also said the cars will eventually listen for sirens and alarms to react to emergency vehicles.

https://twitter.com/x/status/1414436167639093248

Tesla’s camera-based system is an approach that differs from others developing self-driving systems using radar. Earlier this year the company announced the transition to Tesla Vision. Starting last May, Model 3 and Model Y vehicles built for the North American market stopped being produced with radar, instead relying on camera vision and neural net processing for the Autopilot and FSD.

Consumer Reports has reinstated the 2021 Tesla Model 3 with pure vision Autopilot as a “Top Pick” after a top safety designation from the Insurance Institute for Highway Safety.

 

[glide]

“soon”

ROFL

 

[Matias]

Lets see, what Musk said 2019; “he promised that Teslas will be capable of self-driving by the end of this year and self-driving robo-taxis will be on the road in 2020. Also, in two years, the company will be making cars without steering wheels or pedals at all.”

Source

Tesla promises ‘one million robo-taxis’ in 2020 | Engadget

Two days ahead of its first-quarter earnings disclosure, the company had an event to explain its autonomous technology to investors.

www.google.fi

 

[jabloomf1230]

Not THAT hand gesture.

 

[ThomasD]

What about someone flashing their lights for you to go

 

[RayEJ]

Not THAT hand gesture.

Well, maybe even THAT one... <grin>. The question will be our response... ignore? smile? use farticus when passing? just a thought...

 

[qdeathstar]

Not THAT hand gesture.

?

 

[linux-works]

hand gestures? for the car?

I guess they can call them...

toobin signals

(what? too soon??)

 

[father_of_6]

Hey, I take a lot of what Elon Musk says with a grain of salt, too... but I applaud him for trying. While he seems to overestimate what his companies can achieve in a given time, it *still* tends to result in positive outcomes. This overestimation is typical of overachievers.

Maybe FSD is a big failure, but:

• Tesla's cars are great
• The Super Charger network is ahead of everyone else
• SpaceX is kicking butt
• Many people seem to be happy with StarLink

I'm sure there are many more positives to add to that list.

 

[msanghavi]

@diplomat33 does this mean that Tesla cars have a mic as part of HW3 in order to hear sirens? I didn't hear of this before

 

[Carl Raymond]

The cars will listen…

Will this use the existing in-car microphone, or will it require an additional external microphone fitted?

As a mere human, I am sometimes fooled by siren like sounds in the music being played. A clever system will be able to subtract the cars‘ audio from the signal.

Cue the usual concerns of privacy advocates “Tesla is eavesdropping on our conversations“. A few hoops to jump through there.

 

[thewishmaster]

I don't know whether to consider it a good thing that they're working on incorporating it, or alarming that it seemingly wasn't considered until now...

Of course the updated “4D” perception stack with multiple-frame inputs is much better suited for hand gestures and flashing lights on emergency vehicles, but still kind of surprising how far they went along with something that probably just couldn’t handle this at all.

Also the mic input will likely require new hardware with exterior mics because from the Twitter chatter it didn’t seem like there was any plausible way of using existing hardware, including the interior mic (not connected to AP, not to mention all the other noise interference one might encounter inside a car).

 

[diplomat33]

@diplomat33 does this mean that Tesla cars have a mic as part of HW3 in order to hear sirens? I didn't hear of this before

All cars have the internal mic that is used to hear voice commands. I am not aware of the cars having any external mics.

 

[GlockWorkOrnge]

hand gestures? for the car?

I guess they can call them...

toobin signals

(what? too soon??)

Maybe over the heads of some...but I thought it was funny.

 

[GlockWorkOrnge]

All cars have the internal mic that is used to hear voice commands. I am not aware of the cars having any external mics.

That was my question as well. What sensor, other than a mic, will allow a machine to gather and interpret sound data?

 

[Darmie]

All cars have the internal mic that is used to hear voice commands. I am not aware of the cars having any external mics.

Wasn't there something said about if you are on your phone vehicle would hear sounds?

 

[Thp3]

Also the mic input will likely require new hardware with exterior mics because from the Twitter chatter it didn’t seem like there was any plausible way of using existing hardware, including the interior mic (not connected to AP, not to mention all the other noise interference one might encounter inside a car).

Not necessarily. Sirens are designed for people to hear the siren horn while inside the car. So an internal microphone could be capable of hearing a siren. No more hardware is required.

 

[thewishmaster]

Not necessarily. Sirens are designed for people to hear the siren horn while inside the car. So an internal microphone could be capable of hearing a siren. No more hardware is required.

The AP hardware would need to be able to access the mic for that situation, but my understanding is it would need to be wired to AP to use it effectively (eg the way the cameras/radar are wired directly to the AP board). But I don’t know what kind of other voodoo they can cast in order to get mic input from presumably the MCU computer.

 

[Two-rocks]

I’m going to put a siren sound on the boom box. Easy to clear other Tesla’s using FSD/autopilot outta my way…

If another car’s driver waves going by will they call the phantom braking a safety feature?

 

[gearchruncher]

Why is everyone here assuming that this will work on current Tesla cars? They already had HW2, 2.5 and 3, this is probably HW4 knowing how Elon works.

Also, note that "detect hand gestures" does not mean "react to hand gestures."

From the FSD v9 videos, seems like they should be focusing on "detecting huge concrete columns" first.