Visualization issues - 18 wheelers and other large trucks

[jrmnet]

I recently did a trip on a highway with a large number of 18 wheelers and other large trucks. Here are 2 issues that I have run into (M3LR 18,hardware 2.5)

1. 3 lanes in the same direction, where I am in the far left lane (#1) and a truck is in the far right lane (#3). I signal to change into lane (#2). In this scenario I am slightly forward of the truck. I signal to change into lane #2, and start to change into that lane. The UI shows the truck dancing between lane #3 and lane #2 and shows it as red when in lane #2 warning me to not change into that lane. In each case in reality the truck has been straight in lane #3 and never was drifting into lane #2. This is a safety issue, and I have a service center appointment for this.

2. 2 lanes, in the same direction, me in the left lane and trucks in the right lane. Almost every single time I pass a truck, when it gets to roughly the blind spot / rear quarter panel location, on the UI the truck is shown to ping pong between its lane and up against the right side of the visual for my vehicle. After ping ponging a few times it eventually jitters down out of the UI once I fully pass it. Before all of this as I am approaching the truck, the truck visualization is smooth, but once it gets to that rear quarter panel range is when it goes haywire and is no longer smooth and does not represent reality. Again, I would also consider this a safety issue since it looks like an 18 wheeler is trying to play bumper cars with me.

Is anyone else experiencing this? I will report back what I find out from the service center.

 

[jjrandorin]

The visualization on the screen does not represent exactly what the car is seeing, just what its showing you. With that being said, I believe you will get absolutely nowhere attempting to address "My visualization is doing this strange thing" with tesla.

If they tell you something other than a polite (or less polite) "there is no action for us to take, here", I will be beyond shocked.

 

[Maxters]

Wife noticed the same thing.. When vehicles are coming from the rear and passing.. As they transition through the B-Pillar camera the path of travel is not straight.

It looks like the other vehicles are swerving towards you as they go by... Probably just Camera training.

 

[RTPEV]

You should watch the Tesla AI Day presentation, and specifically the part(s) about the temporal handling and camera "meshing" (I can't remember the exact term they used) of the neural net. The NN will handle this much better with future builds as the computer will have a "memory" and be able to mesh together the various camera views to recognize large trucks more accurately, versus the current single frame/independent camera object recognition which has a hard time with objects bigger than a normal vehicle, and particularly ones like semi trucks that might be "empty" underneath as you pass.

As far as this being a safety issue, I would certainly be concerned with it if it was recognizing the truck (in scenario #1) as being in lane #3 (rightmost) but in reality was in lane #2. In that case your car might attempt to pull into an occupied lane #2, which would not be good (although I suspect even in that case as the car approached the truck, the object recognition would improve and it would abort the lane change). It seems like it is taking the conservative approach by NOT allowing the lane change. And yes, I have seen this behavior. It is annoying both to me (because the car attempts to pull right and then aborts), and I'm sure to the cars behind me. But I'm not sure I would count it as a safety issue, particularly since I am monitoring what is going on and if/when this happens I will disable Autopilot and do the lane change myself.

Scenario #2 is definitely not a safety issue. That's just a visualization quirk and has nothing to do with the vehicle's operation, unless you are saying that you have experienced some kind of evasive action the car took. In my experience I have never seen this.

 

[jrmnet]

You should watch the Tesla AI Day presentation, and specifically the part(s) about the temporal handling and camera "meshing" (I can't remember the exact term they used) of the neural net. The NN will handle this much better with future builds as the computer will have a "memory" and be able to mesh together the various camera views to recognize large trucks more accurately, versus the current single frame/independent camera object recognition which has a hard time with objects bigger than a normal vehicle, and particularly ones like semi trucks that might be "empty" underneath as you pass.

As far as this being a safety issue, I would certainly be concerned with it if it was recognizing the truck (in scenario #1) as being in lane #3 (rightmost) but in reality was in lane #2. In that case your car might attempt to pull into an occupied lane #2, which would not be good (although I suspect even in that case as the car approached the truck, the object recognition would improve and it would abort the lane change). It seems like it is taking the conservative approach by NOT allowing the lane change. And yes, I have seen this behavior. It is annoying both to me (because the car attempts to pull right and then aborts), and I'm sure to the cars behind me. But I'm not sure I would count it as a safety issue, particularly since I am monitoring what is going on and if/when this happens I will disable Autopilot and do the lane change myself.

Scenario #2 is definitely not a safety issue. That's just a visualization quirk and has nothing to do with the vehicle's operation, unless you are saying that you have experienced some kind of evasive action the car took. In my experience I have never seen this.

Yes, I'm more concerned with scenario #1 as far as safety is concerned.

However with scenario #2 if a large truck in reality did start to drift into my lane when at the rear quarter panel position, my mind would probably be trained to ignore it assuming it was just the UI quirk.

I do hope that both of these issues will eventually addressed with a future update.

 

[jjrandorin]

I do hope that both of these issues will eventually addressed with a future update.

I think they did, actually... but in HW3. Since I purchased FSD with my 2018 model 3, I also had HW2.5 and saw all that bouncing / drifting etc that was REALLY bad in the UI, then got a bit better but still wasnt great. When HW 3 became available for my vin and I got that installed, I saw a marked improvement.

In other words, I expect you will not see much improvement over the visualizations in HW 2.5 now, or going forward, and any improvements would be relegated to HW3 and above. Thats just speculation but I feel its a valid point of view, based on how tesla (and other tech companies) have operated in the past.

 

[jrmnet]

I think they did, actually... but in HW3. Since I purchased FSD with my 2018 model 3, I also had HW2.5 and saw all that bouncing / drifting etc that was REALLY bad in the UI, then got a bit better but still wasnt great. When HW 3 became available for my vin and I got that installed, I saw a marked improvement.

In other words, I expect you will not see much improvement over the visualizations in HW 2.5 now, or going forward, and any improvements would be relegated to HW3 and above. Thats just speculation but I feel its a valid point of view, based on how tesla (and other tech companies) have operated in the past.

Thanks, this is good to know and I suspected as much. I remember early on the dancing cars was a big issue, and for me it has been mostly resolved except for the issues listed above which seem similar.

Actually there is 1 additional issue UI visualization issue I experience on a regular basis

3. When a car is directly behind me, on the UI it constantly blinks on and off

 

[jjrandorin]

Thanks, this is good to know and I suspected as much. I remember early on the dancing cars was a big issue, and for me it has been mostly resolved except for the issues listed above which seem similar.

Actually there is 1 additional issue UI visualization issue I experience on a regular basis

3. When a car is directly behind me, on the UI it constantly blinks on and off

I still see that sometimes on HW 3 as well. I think that falls back to the visualization not showing you exactly what the car is seeing, exactly when the car is seeing it, at all times.

 

[jrmnet]

Per Tesla rep, after running diagnostics on my 3, all cameras are working as expected within spec. Hopefully a future update fixes this.