Suspected repeater camera defect that affects FSD performance

[S4WRXTTCS]

My co-worker got their 2022 Model Y last week. We have compared AP performance and it is the same with both new and old cameras.

Do both of you have the auto-lane change feature? As technically it would be AP with FSD (or EAP) performance.

How many auto lane changes at night did you do?

It's going to be really hard to do a comparison as you'd have to do hundreds of lane changes at night, and then tally the difference in how many executed correctly.

There is also a variation in how bad the issue is, and variation on how much this defect impacts the auto-lane change.

For example on a recent trip back from Portland on a clear night I had zero issues doing auto-lane changes to the right, but sometimes it wouldn't allow me to do auto-lane changes to the left. To the human eye both cameras had the same level of obstruction due to glare.

Visually inspecting both cameras revealed that they were both clean of any dirt or debris.

My plan is to do before and after the fix tests. I'll be irked if Tesla has me pay for the new cameras, but I absolutely love the autolane change feature. So I'll totally pay for the upgrade if there is a good chance it will make it better. Even if it doesn't it still makes it easier for me to see, and it makes the dashcam videos look better if I ever need to use it to prove something.

 

[STS-134]

Was this because the function is equivalent or because you did not encounter a situation in your testing that would be affected by the defect?

Computers can easily see through glare like that. But it still has to be a design flaw. I highly doubt the design team had this idea that they were going to design cameras that pick up glare from the turn signals because the software guys can easily filter out the glare.

 

[S4WRXTTCS]

Computers can easily see through glare like that. But it still has to be a design flaw. I highly doubt the design team had this idea that they were going to design cameras that pick up glare from the turn signals because the software guys can easily filter out the glare.

The issue with "the software guys can easily filter out" is that there is a limit to what filtering/neural networks/etc can handle.

At night you have this glare, the glare from headlights themselves, and rain.

You start adding things up and its a terrible image that not even human brains can interpret what its in the frame. That's not even getting into estimating the distance it is away.

Auto-Lane change is a really tough thing to get right especially with Vision only.

Tesla has committed themselves to Vision only so maybe they should use some of the money saved by not having Radar to fix the cameras so the view is the best it can be.

 

[S4WRXTTCS]

No one really knows if the glare causes issues for AP & FSD except Tesla. Neither FSD nor AP are perfect. Is it possible that some of the mistakes they make are from the cameras? Yes. Can anyone prove it? No.

If we had enough volunteers with both pre-fix, and post-fix cameras we could do an experiment to gather data.

Where we compared the completion performance of a manually initiated auto-lane change done at night.

Maybe 50 vehicles of both types, and 100+ lane changes from each car.

Verifiable data where each person saved the dashcam of the each attempted lane change. That way we could rank each cameras image and see if there is any correlation with performance.

Only problem with this is we'd have to find 100 really OCD people to gather all this data.

I'd put money on the camera defect impacting the auto lane change. Maybe we could use the betting money to pay the 100 peoples for their work? That's probably not needed as they're Tesla owners so their used to doing work for free.

 

[MP3Mike]

You start adding things up and its a terrible image that not even human brains can interpret what its in the frame. That's not even getting into estimating the distance it is away.

Go back and review the videos shared earlier in this thread. What the FSD computer gets is not the same as what is displayed to the driver. It has way less glare and in some cases the light from the turn signal actually improves the image more than the glare harms it. (Though it would probably be even better without the added glare.)

 

[Ruffles]

We both have full FSD. I'm in the beta (yes, I know we have different SW). Driving around at night, both cars behaved the same. They changed lanes, identified surrounding cars etc. There was no obvious difference. Did we test ever possible condition? Of course not. There is no reason to. In addition to comparing with my co-workers car, my car behaves the same day or night so that again tells me there is no impact from the glare because during the day in the bright light, it doesn't exist. Finally, I've not seen any reports from the people who did replace their cameras that suddenly FSD and AP are better than before.

Seriously, you all are chasing a problem that doesn't exist.

 

[S4WRXTTCS]

Go back and review the videos shared earlier in this thread. What the FSD computer gets is not the same as what is displayed to the driver. It has way less glare and in some cases the light from the turn signal actually improves the image more than the glare harms it. (Though it would probably be even better without the added glare.)

I agree with the assessment that we can't use what we see because obviously its gone through image processing intended for human viewing.

This is something we'd just have to test to see what the test results show.

 

[sleepydoc]

Go back and review the videos shared earlier in this thread. What the FSD computer gets is not the same as what is displayed to the driver. It has way less glare and in some cases the light from the turn signal actually improves the image more than the glare harms it. (Though it would probably be even better without the added glare.)

Link? I didn't see any videos, only pictures and regardless, how do you know what the FSD computer is actually getting? The only people who definitively know that to my knowledge are Tesla engineers.

There are a lot of reasonable arguments for and against, but I haven't seen any definitive proof either way. All of the arguments on both sides involve assumptions. Take the statement that 'the computer can filter out the glare' - maybe, but it depends on the hardware. Many camera modules have built in accommodation for varying light levels meaning the computer effectively gets data that's already processed by the camera module. On the other hand, it's possible that the cameras are essentially HDR cameras and the computer does get enough data from the raw image to effectively process.

Computers can easily see through glare like that. But it still has to be a design flaw. I highly doubt the design team had this idea that they were going to design cameras that pick up glare from the turn signals because the software guys can easily filter out the glare.

See my statement above. It depends on the hardware, the camera module software and the software of the central computer.

 

[S4WRXTTCS]

Seriously, you all are chasing a problem that doesn't exist.

For me its trying to determine the root cause of an existing issue.

Why do my manually initially auto-lane changes fail more often at night? Why is it only in one direction, and not the other.

To me the camera defect is a plausible explanation.

I like data so before I get it fixed I'll collect my own data of 100+ auto lane changes at night, and then compare that data to auto-lane changes after the cameras are replaced.

It's not an easy comparison as environmental conditions will obviously differ.

But, that's the case with so many Tesla things. For example Tesla vision seems more consistent during auto-lane changes during the day than before Tesla Vision. Where it doesn't get tripped up so much by semi-trucks in the lane next to the one I'm auto lane changing into. Is that really true? Or just a premature conclusion without much supporting data?

 

[MP3Mike]

Link? I didn't see any videos, only pictures and regardless, how do you know what the FSD computer is actually getting? The only people who definitively know that to my knowledge are Tesla engineers.

Sorry it wasn't in this thread. My understanding is that they captured the raw signal that goes to the FSD computer...

https://twitter.com/x/status/1503184635915735044

 

[Graham J]

Sorry it wasn't in this thread. My understanding is that they captured the raw signal that goes to the FSD computer...

https://twitter.com/x/status/1503184635915735044

I hadn’t seen that; it does look promising. It’s surprising these photon counts being thrown off by a local light source aren’t an issue but I’m not a NN expert.

It begs the question why they aren’t better processing these data for display on DashCam and the blind spot viewer. If this can be improved with a software update rather than camera replacement I’d be satisfied with that.

 

[nvx1977]

It begs the question why they aren’t better processing these data for display on DashCam and the blind spot viewer. If this can be improved with a software update rather than camera replacement I’d be satisfied with that.

Dashcam was a bare-bones implementation. Tons of people had compatibility issues with USB sticks or just getting it formatted exfat. The playback interface was and still is atrocious. Video streams were garbled sometimes. All indications is that they quickly pulled something together and didn't invest a lot of time into it.

It's a super useful feature, so I wish they would spend more time on it. But like Spotify and other features in the UI, it's pretty clear they built an MVP (minimum viable product) and then never bothered to refine it much after that.

 

[nvx1977]

Sorry it wasn't in this thread. My understanding is that they captured the raw signal that goes to the FSD computer...

https://twitter.com/x/status/1503184635915735044

I mentioned this earlier in the thread, but when a car is approaching, the headlights from the other car causes the camera to reduce its exposure sensitivity, which reduces the glare effect. I have seen this on my car many times. It is when a car is approaching where we care most that our car doesn't move over.

For me, I have never had any problems with auto lane changes or manually initiated changes at night on any version of FSD beta. Pre beta, I had issues with the car sometimes being indecisive and canceling the lane change in midstream, but that would happen day or night, so the root cause likely wasn't this glare issue. That issue got better over time (bad when NoA first released; better by late 2019).

From my perspective, the "design defect" here is pretty much cosmetic. I see no evidence that it's affecting FSD performance in any way.

 

[sleepydoc]

Sorry it wasn't in this thread. My understanding is that they captured the raw signal that goes to the FSD computer...

https://twitter.com/x/status/1503184635915735044

Thanks for that. I didn’t see in the thread how he got the images. Was he able to hack into Tesla’s system to get the raw stream?

Regardless, I agree with @Graham J - if they have a better image, why didn’t they bother to take the time to put it on the screen. It speaks quite poorly of their UI team.

 

[drtimhill]

Correct. Tesla added a feature that does not work correctly on cars pre-2021.

But you have taken it on yourself to define "correctly" as "not the way I want it". Tesla can quite easily fix this .. they can disable the feature entirely on cars that do not have the newer camera modules. It seems that based on your criteria that would satisfy you (though it would certainly not satisfy me or many others here). Correct?

 

[drtimhill]

FSD + EAP
DashCam
Manually selected Camera view where you see the side rear facing cameras along with rear camera
The newly added camera view on lane change

In all cases it impacts the quality of the feature that relies on it.

Apart from your opinion, do you have any quantitative evidence that FSD/EAP are impacted by the glare? And please don't say "its obvious from the screen that it must be because I can't see through the glare", because that is irrelevant.

 

[drtimhill]

I bought it used after the update was made available. The issue isn’t so much the camera view itself but the fact that the module has a defect. I paid a large sum of money for this car and feel it’s reasonable to expect all of its systems to function properly even if I would have been unable to tell they weren’t at some point in the past.

Anyway whatever, I don’t actually care that much at this point but that’s the topic here so that’s what we’re discussing.

But the basic argument remains .. at some point the car was upgraded (for free) to provide side-view cameras. It's immaterial if you got the car before or after that happened. Because any argument about the cameras not being up to the job can be interpreted in one of two ways:

(1) Tesla needs to upgrade everyones cameras.
(2) Tesla should have only enabled the side-view cameras on newer cars with the new camera modules.

So, your arguments can be interpreted as you saying "they should not have made this feature available on my car since the hardware, in my opinion, does not support it properly". And when viewed like this, the solution is easy. Switch the feature off! Then you will be happy, since you no longer have improperly functioning side-view cameras.

Of course, this is not what you want. What you want is option (1), but I cannot see any argument that option (2) is not equally valid. Sure, you got a car with not-very-clear nighttime side-view cameras, and newer cars have better ones. So what? My M3 doesnt have a heated steering wheel, or powered trunk, which were added later .. should I go demand them free from Tesla?

 

[drtimhill]

Does a software feature upgrade imply an obligation to upgrade the hardware? No other manufacturer does software and feature upgrades like Tesla does so there's no precedent for the current situation.

It may be that Tesla only tested the feature on newer cars and was unaware of the issues until after release but they could have avoided a lot of this by issuing a warning with the software on older cars when you activate the feature stating there 'may be glare in some models d/t hardware incompatibilities.'

It seems to me that Tesla probably knew the cameras were not great for human viewing. They then had a choice: (a) only enable the feature on newer production cars since the older cameras would give mixed results or (b) enable the feature for everyone since they work well in the daytime and its a free thing to everyone anyway. I'm glad they chose (b), since I get side view cameras that work fine most of the time, but if Tesla read stuff like this and demands from older owners for free upgrades they are going to think twice next time about going down path (b) .. so we all potentially get screwed because a small minority are demanding freebies from them.

I think the appropriate phrase is "Dont look a gift horse in the mouth."

 

[Graham J]

But the basic argument remains .. at some point the car was upgraded (for free) to provide side-view cameras. It's immaterial if you got the car before or after that happened. Because any argument about the cameras not being up to the job can be interpreted in one of two ways:

(1) Tesla needs to upgrade everyones cameras.
(2) Tesla should have only enabled the side-view cameras on newer cars with the new camera modules.

I only stated that I bought it used because the person I responded to said the feature didn't exist when I bought the car, which isn't correct. I agree it's not relevant to the argument about the module being defective.

There's no need to interpret my argument because I've stated it outright: Defective equipment should be replaced regardless of whether I am able to detect the problem. For me this wasn't about the camera view looking bad it was about the module not working properly.

However in light of the raw output actually looking pretty decent - even with the light bleed - my opinion now is that the tone mapping that occurs when the raw output is compressed and sent to the MCU is what needs correction. The repeater module seems to be good enough.

 

[Scotty7]

While no doubt everyone will howl "fan boy" at me, I'm a bit confused by your logic. When you bought the car it didn't have side view cameras at all. Correct? Now Tesla add the feature for free, but it doesnt work as well as you want, and you demand that they upgrade your hardware (again for free) so it meets your expectations?

I really do not care for the performance of the blinder-view-popup or the sideview camera view (partly because V11 is so badly designed that my hands block the view when it's relevant anyway), the problem is if it affects FSD, which I have paid several grand for and seen next to no return on living in the UK.

We've been around the circle of debating If this is going to impact FSD / autopilot, which we can't really come to a conclusion on. I agree Tristan's raw feed footage is promising. If it turns out this does notably affect FSD performance, I expect the cameras to be replaced FOC given that fee is supposed to include hardware upgrades (and because I have documented assurance from Tesla service staff if camera performance isn't sufficient enough for FSD they'll be replaced).

Regardless, it sucks they've been replacing light-leaking cameras FOC under warranty seemingly at random, while simultaneously giving garbage excuses why they're not going to do that for the other half of people asking.