Tesla.com - "Transitioning to Tesla Vision"

[greenmore]

The problem with 2 different inputs is how do you know which one is right?

If you think the camera is good enough to trust then why bother with the radar?

If you think the radar is right, well, you get exactly the phantom braking you get today.

This is a very naive view of sensor fusion. It's not a "which one do you choose" problem. Elon tweeted this as well and it immediately reminded me of the old guy in congress who described the Internet as a series of tubes.

Sensor fusion doesn't do binary picks. It's getting simultaneous inputs with known noise statistics and it weights different inputs based on how well the captured data fits into the statistics. On data with lower dimensionality you can use a Kalman filter but there are more sophisticated solutions for more complex data.

 

[greenmore]

Therein lies the problem, if the camera can tell it's a bridge what's the point of the radar?

There's basically 4 scenarios:
1) Both sensors determine there is something in its path: you have high confidence there is.
2) Both sensors determine there isn't something in its path: you have high confidence there isn't
3) radar determines there is something in its path, but camera doesn't (basically the situation you point out at end and also in phanthom braking events): if you want to play it safe, you brake or slow down
4) camera determines there is something in its path, but radar doesn't: same deal, if you want to play it safe you brake or slow down.

The contribution to phantom braking is largely between which plays a bigger role: #3 or #4? And #3 really is where radar plays its role in improving safety, but also where it can contribute to phantom braking or "noise" overall even if car decides not to act.

I don't see it phantom braking as a huge problem. Frankly, I don't know why people are complaining about it.

In most cases radar helps identify hazard situations when vision isn't yet able to recognize them. Like vision wouldn't be able to detect that you are driving towards a featureless wall if that has lanes painted on it. Basically a classic Wile E. Coyote situation. Radar is there for the rescue.

I see radar as a fail-safe for cases where vision is not sophisticated enough yet to correctly assess the situation.
With every new version of the vision system, phantom baking will occur less and less frequently, but it doesn't mean radar is useless in the future.

 

[texas_star_TM3]

This is a very naive view of sensor fusion. It's not a "which one do you choose" problem. Elon tweeted this as well and it immediately reminded me of the old guy in congress who described the Internet as a series of tubes.

Sensor fusion doesn't do binary picks. It's getting simultaneous inputs with known noise statistics and it weights different inputs based on how well the captured data fits into the statistics. On data with lower dimensionality you can use a Kalman filter but there are more sophisticated solutions for more complex data.

a real life example on why *more* sensor input and redundancy in sensors is important would be the Boeing 737 MAX ...

 

[texas_star_TM3]

I don't see it phantom braking as a huge problem. Frankly, I don't know why people are complaining about it.

In most cases radar helps identify hazard situations when vision isn't yet able to recognize them. Like vision wouldn't be able to detect that you are driving towards a featureless wall if that has lanes painted on it. Basically a classic Wile E. Coyote situation. Radar is there for the rescue.

I see radar as a fail-safe for cases where vision is not sophisticated enough yet to correctly assess the situation.
With every new version of the vision system, phantom baking will occur less and less frequently, but it doesn't mean radar is useless in the future.

I'd rather experience occasional phantom brake and not slam into an object the cameras detect too late in the worst case ... *nobody* here can make the argument that at night or in pouring rain camera based vision reaches farther than the conti radar they just removed from the cars

 

[gearchruncher]

a real life example on why *more* sensor input and redundancy in sensors is important would be the Boeing 737 MAX ...

Not sure this is a good example. All the 737 Max MACS needs is accurate Angle of Attack data. The failure here was that the sensor actually failed, not that it is a sensor which is confused in some situations. In the 737's which had dual, identical AoA sensors, they never had a system failure.

Redundancy != Dissimilarity. Radar and cameras are dissimilar, which does give you some redundancy, but they are different things, and what we really are discussing in this case is not where the cameras fail to do their job (photons to signals), but places in where those signals are insufficient or complex to do wat the system needs to.

 

[qdeathstar]

I actually used the specific image on purpose. The Stop sign on a bus is visible from the side even when it's not flipped out. This can be the image you see of a bus stopped in traffic on a cross street as you approach it.
A full FSD system has to understand a LOT of context for signs, not just identify their existence visually.


If stopping for a school bus isn't common enough to drive automated training, then that just tells us how little we can learn from driving around and how long the tail is. I personally probably have to stop for a school bus 50 times a year because of how my commute lines up with bus routes.

Even in that scenerio, wouldn’t you want the car to stop for that stop sign in leu of plowing into that bus?

 

[stopcrazypp]

So if Tesla was so forward looking to realize this would mean the check marks were lost, and that would cause distraction, and they are super sure they are going to solve this in vision in a few weeks after 3 years of work, but they also still had plenty of radars, then:

There are 4 parts in that statement:
1) knowing check marks will be lost
2) knowing that would cause distraction
3) being "super sure" it would solve in a few weeks after 3 years of work
4) still had plenty of radars.

#1 they lost check marks before back in the AP2 transition, so I think it's a given they already know they would lose check marks here also.
#2 it's almost common sense they would know losing check marks would have negative impact, and if it's not clear before, the recent reporting would make it doubly clear. Again, they had the experience of AP2 also.
#3 We don't know yet how long it will take to reach parity. Tesla's official statement says "In the weeks ahead, we’ll start restoring these features via a series of over-the-air software updates", but they don't have a solid timeline yet when that will be finished.
#4 we know they have enough supply of radar for delivery for other markets and models. We also know they have been steadily raising prices on the 3/Y in recent weeks, and also have even cost cut on the passenger lumbar (there may be other changes we are unaware of). The cost of the radar is much higher in comparison and definitely something Tesla may see incentive to remove as soon as possible if they deem it unnecessary, regardless of availability.

1) Why did they do this at all, instead of waiting for vision to be at parity? All for $75 per car for "the upcoming weeks"?

As above, we don't know yet how long it will take to reach parity, and given they are even removing a part that costs much less (passenger lumbar controls), the radar cost is nothing to scoff at.

2) Why did they not talk to NHTSA until the day after they made this public?

We don't know what date they talked to NHTSA. NHTSA only said they were briefed on it after the media asked the day after, they did not say when Tesla did so. So we only know Tesla briefed them before NHTSA was asked by the media, we don't know how long before.

3) If they are sure that this will still meet AEB requirements, that means it's been tested at least internally. Why couldn't they meet the requirements in other countries with this change, especially since shipping to other countries takes weeks. Why not "double down" here like they did for US customers? If Tesla is a safety first company, this needs to be a parity or safer system than the previous one, which they should be able to argue to any of those evil regulators.

It may meet AEB requirements here but that doesn't necessarily mean it meets them elsewhere, as the test standards are different. From the discussion about NHTSA's test standard, we already know it is not harmonized with Euro NCAP. The testing schedules may also be different (we can't assume the various governments will immediately schedule a test for a mid year change, it may take much longer than that).

4) Why did they scrub the blog post explaining how radar was helpful instead of updating it to say "we've learned since then"? I thought they were proud of their vision superiority.

Not sure what this part has to do with the above points? This may simply be to "save face" for the company given inconsistency with the current message, but it does not have an effect on the above 4 points you mentioned.

5) Why didn't they make this change and only apply it to new orders, and deliver people that ordered with Radar the cars they ordered, instead of going back and changing a configuration on them, requiring acceptance of this change because they needed to remove advertised features. Just slap a radar on any VIN ordered before date X.

As above, doing the change as soon as possible saves them the most money, while they are continually raising prices to make up for rising costs (I'm guessing now it has reached back to previous prices, Tesla is looking to cutting some costs instead of continual $500 increases). That's how Tesla had operated, making immediate changes to hardware when they deem fit. There was never a concept that you "lock down" the specs at order (you can ask the people who ordered Model Y LR RWD how well that concept worked for them), it was always what is in the car at delivery that was most important.

Arguing that Tesla did this for any reason besides a shortage requires simultaneously saying Tesla is super smart and super dumb at the same time. And the only reason to do that dance is to defend Tesla not being transparent about why they did this.

See above. You are putting multiple elements together in what seems like a false dilemma.

 

[dgatwood]

Not sure this is a good example. All the 737 Max MACS needs is accurate Angle of Attack data. The failure here was that the sensor actually failed, not that it is a sensor which is confused in some situations. In the 737's which had dual, identical AoA sensors, they never had a system failure.

As I understand it, prior to the recent software fix, the MCAS system got data from only a single AoA sensor. I think they all had two installed. The software just ignored one. There's still a problem, though, which is that when they disagree, all it can do is warn the pilot, who then has to try to figure out which one is lying. That's why there's pressure to force them to add a third AoA sensor.

There was one difference between planes that contributed, though — the last flight that crashed lacked an AoA disagree light that would have told the pilot that one of the AoA sensors was misbehaving, which might have made the source of the bad data more obvious to the pilots, and thus might have led them to disable MCAS more quickly. But there are a lot of "might"s in that sentence.

 

[gearchruncher]

Yep, so they knew they would lose check marks from all the previous times they did this, and had plenty of time to plan for this. Knowing this, they:

1) Did not include this "temporary" reduction in safety ratings in any of their messaging ahead of it happening. Instead, allowed it to become a news story when NHTSA removed it and had no prepared message for this and had to react. Even though they had a very detailed "transitioning to Tesla vision" page with all kinds of FAQs. When they did add something 3 days later, they added this, which is still there:

Does the transition to Tesla Vision affect Model 3 and Model Y safety ratings for vehicles delivered on or after May 2021?
No. The transition does not affect Model 3 and Model Y crash safety ratings. Vehicles equipped with Tesla Vision retain the same crash safety ratings as vehicles equipped with radar.

Whoops. Doesn't seem like the kind of thing you'd say when you knew your safety ratings were about to change in a big news cycle. Yes, I am aware it says "crash safety" in the answer, but the question just says "safety ratings" which absolutely changed. It's either slimy, or they had no idea it was about to happen, except they did know it was going to happen because it already had.

2) Did not tell anyone when NHTSA would retest the car, until it was a news story and Elon tells Elektrek of all places that "he checked with the AP team and AEB is still there, and they're testing it next week" (which is now this week).

2.5) Elon wasn't sure if AEB was removed until he checked with the AP team days after NHTSA removed the ratings.

3) Decides to remove it on the 3/Y, but not S/X, so now they get TWO news cycles of it being removed when (if ever...) they do it on the S/X, instead of one.

4) Needed to update their website multiple times in one day, piecemeal, and then kept editing it over the next week. Even removed AEB for a few hours until Reddit noticed. Yep, well planned. They've updated the transitioning page at least 50 times since it was first posted.

So yep, they're either still complete idiots that can't plan at all, or they ran out of radars. Which one is a better message?

There was never a concept that you "lock down" the specs at order (you can ask the people who ordered Model Y LR RWD how well that concept worked for them), it was always what is in the car at delivery that was most important.

I cannot believe someone would use this argument to defend Tesla. The idea that you have no idea what you are getting when you order a car, and you have to go over it with a fine toothed comb at delivery because you should know that anything on the web or order form is just hopes and dreams? Meanwhile, most Tesla defenders say "that wasn't on your order page, you aren't owed that!"

All of this just so you can avoid acknowledging that Tesla has supply chain issues just like everyone else and that's why they removed Radar from only some of their products. And Lumbar.

 

[Yoliber]

If AP still requires autowipers and auto high beam to reach parity, would you guys consider it parity? Auto high beams make night AP pretty unusable to me.

 

[qdeathstar]

If AP still requires autowipers and auto high beam to reach parity, would you guys consider it parity? Auto high beams make night AP pretty unusable to me.

i would. But that also would mean those two features would have to work.

 

[techlogik]

Can you post where you seen reports of radar shortages? There are chip shortages (which affects ADAS option packages), but nowhere did I find reports of radar shortages. Someone mentioned BMW said there was a radar shortage, but when I asked for a source for that claim, there was no response.

It’s a lie. We know this because all cars outside the US are still getting radar and the S and X are in the US.

 

[Knightshade]

.

This is a very naive view of sensor fusion. It's not a "which one do you choose" problem. Elon tweeted this as well and it immediately reminded me of the old guy in congress who described the Internet as a series of tubes.

Right.

The dude who lands reusable rockets doesn't understand science.

That makes sense.

It’s a lie. We know this because all cars outside the US are still getting radar and the S and X are in the US.

...what?

A shortage means less of, not none of.

There's a shortage of GPUs right now too, that doesn't mean nobody is getting any... it just means fewer people are.

If you had a shortage of radars removing them from the 2 models you make the most of, in the country you sell the most of them in, would be exactly the first place you'd remove them.

 

[S4WRXTTCS]

It’s a lie. We know this because all cars outside the US are still getting radar and the S and X are in the US.

To lie you have to state something or lie by omission.

Has Tesla stated any official reason as to why the Model 3/Y have started to ship without the radar? Only thing I've seen is that the radar isn't needed, but obviously this doesn't explain why Tesla didn't wait until Tesla vision was at parity. Or why the Model S, and X in the US still include the radar.

So I don't rule out Tesla lying, but just not the way you stated.

We do know Tesla is lying about the lumbar support. They are lying because they're stating a statistic that doesn't tell them what they think its telling them. They're claiming lumbar support isn't used much by the front passenger when the front passenger has no reason to adjust lumbar once its set. Tesla is full of smart people so they obviously know this. They just think they can trick us into thinking its true.

 

[powertoold]

They're claiming lumbar support isn't used much by the front passenger when the front passenger has no reason to adjust lumbar once its set. Tesla is full of smart people so they obviously know this.

Tesla should also know the lumbar setting as well, not only the frequency of adjustment. It’s possible that the lumbar setting was default in most cars, meaning few people actually adjusted it.

 

[S4WRXTTCS]

If AP still requires autowipers and auto high beam to reach parity, would you guys consider it parity? Auto high beams make night AP pretty unusable to me.

How its thought of will likely have a lot to do with how Tesla handles HW3 cars with Radar versus how Tesla handles HW3 cars without Radar.

If Tesla stops using the radar even in cars with Radar then they're at parity with that firmware version regardless of functionality or performance. They they just downgraded or upgraded everyone depending on perspective.

But, if Tesla continues to use data from Radar in some useful manner than they're never going to be at parity.

 

[MP3Mike]

We do know Tesla is lying about the lumbar support. They are lying because they're stating a statistic that doesn't tell them what they think its telling them. They're claiming lumbar support isn't used much by the front passenger when the front passenger has no reason to adjust lumbar once its set.

No we don't know that. Elon didn't say that it wasn't changed very often, he said "Logs showed almost no usage." That may mean that most passenger seats never had the lumbar adjusted even a single time. And that wouldn't surprise me given the number of people that didn't know that the driver or passenger seats had adjustable the lumbar support option until the news of it being removed from the passenger seat broke.

 

[orion2001]

Right.

The dude who lands reusable rockets doesn't understand science.

That makes sense.

This may come as a surprise to you, but Elon Musk isn't a rocket scientist. Nor is he a Machine Learning expert. In fact, as a ML researcher with a fair bit of experience, I will go so far as to say that he understands very little when it comes to ML and most of his tweets are nonsensical technobabble and exhibit a very poor grasp of how ML systems work and what is actually achievable given current technology and the computational budget available to his cars.

Look no further than the awkward events where he shares the stage with Andrej Karpathy (someone who really knows what he is talking about) and Andrej is constantly cringing at the hyped up nonsense being pitched by Elon for proof. Well, that and his FSD is a solved problem statements and his proclamations of true autonomous driving always being just around the corner.

He is a smart man, no doubt. A visionary even. But he isn't a rocket scientist, and he most certainly is terrible at anything machine-learning related.

Sensor fusion isn't necessarily easy, but it also isn't an either-or proposition. The larger issue imo is that they just have a very subpar radar system compared to most other auto manufacturers. It isn't clear if that is due to technical incompetence in processing radar data (it can be tricky and easy to screw up. We process a lot of radar data for other applications at work and have seen truly horrible beamformed radar imagery from some vendors/clients that we were able to significantly improve), or just poor hardware, or some combination thereof. Plenty of other manufacturers use radar for TACC and phantom braking isn't a thing for them and somehow is only unique to Tesla, but radar gets to be the scapegoat for it.

 

[Ramona48]

drove through miserable weather in Texas yesterday and the auto-wipers are either going absolutely crazy (wiping at a speed the windshield is dry and they begin to drag across dry glass) - or - they kick in way too late and not before the windshield is covered in rain. I love the car but the vision windshield camera system is nowhere near the performance of a proper rain sensor in most other vehicles...

 

[Knightshade]

This may come as a surprise to you, but Elon Musk isn't a rocket scientist.

It would if it were correct.

Elon is literally the chief engineer for SpaceX.

If you'd like even more evidence someone conveniently compiled it here-

https://www.reddit.com/r/SpaceXLounge/comments/k1e0ta

It includes quotes from numerous high level aerospace engineers from JPL, NASA, etc, numerous long-time space reporters, and even astronauts, all confirming this is not a vanity title.

Look no further than the awkward events where he shares the stage with Andrej Karpathy (someone who really knows what he is talking about)

It's funny you mention Karpathy since he also has been saying for years now they're designing the system to operate purely on vision

So make up your mind- is someone saying that an idiot who describes the internet as a series of tubes- or someone who REALLY knows what he's talking about?