Model 3 Object visualisation

[Andywil]

Have noticed that my car consistently shows road bollards as traffic cones, which is maybe acceptable.
What is very odd is that today I was near a skip with 3 cones next to it, the cones were shown but the skip (size of a small car) seemed invisible.
I hate to think what the consequences of this could be if roadside skips are not seen as existing.

 

[MrBadger]

I think traffic cone visualisation serves two purposes.

First, they are an indication to the driver that the car has recognised that that road segment is possibly not as mapped. Whether the car acts on this information more than offering a lane change away from cones is not obvious.

Second, it is a way of Tesla saying ‘look what we can see’ and more importantly, where we can see it. In my experience, whilst traffic visualisation can be a bit hit and miss, positioning of traffic cones in 3D space is imho a pretty decent showing even if it doesn’t matter that their orientation is off. The same cannot be said of some vehicle visualisations, especially orientation and following vehicles that can be massively out.

 

[Glan gluaisne]

Mine spotted a Liberal Democrat sign at the side of the road this morning, and flagged it up as a traffic cone. Wasn't sure if it was a visualisation error or a political comment...

 

[Adopado]

Mine spotted a Liberal Democrat sign at the side of the road this morning, and flagged it up as a traffic cone. Wasn't sure if it was a visualisation error or a political comment...

excellent! Mine spotted a traffic cone that was on it's side and behind sign at the side of the road! The resulting avatar representation was of an upright cone of course, but it really does seem to have an eye for a cone!

 

[Andywil]

My main concern was the complete lack of awareness of the skip which is a serious piece of metal and not one I would like to collide with.
If a car is recognisable why not a skip?

 

[rotor2k]

It’s only showing you things it recognises. It doesn’t mean it doesn’t see an obstacle, it just doesn’t know how to visualise it for you.

 

[Roy W.]

Tonight coming back from Birmingham we came along a stretch of the A38 with roadworks, and many, many cones. It was entertaining to see them all on the car screen.

I was on Autopilot for most of that stretch, and it coped admirably until we came to a stretch where the inside lane that I was in was closing, and the cones were taking everyone across into the outside lane.

Unfortunately Autopilot, although it could visualise the cones in front, decided it was going to stay in the inside lane, until I did a little swerve to disengage Autopilot and manually drive through the roadworks!

But FSD will be feature complete by the end of the year...

 

[MrBadger]

Unfortunately Autopilot, although it could visualise the cones in front, decided it was going to stay in the inside lane, until I did a little swerve to disengage Autopilot and manually drive through the roadworks!

But FSD will be feature complete by the end of the year...

FSD would have prompted you to move over if it was running adjacent to a lane of cones - it cannot yet do unconfirmed lane changes, although can take off ramps, so is unable to do the lane change automatically.

Did the whole lane move over, or did the inside lane close? If former, I have seen the car treat cones as a lane marking and follow them, but would not trust it in all circumstances. If the lane just closed down, then I think AP was doing its job of keeping you in lane although it is unclear what other safety alerts/actions this scenario would have triggered. Certainly I would have thought that a merge was outside basic AP functionality and a scenario where driver takes control.

Would be interesting to see how it would have behaved in a FSD equipped car.

 

[Roy W.]

FSD would have prompted you to move over if it was running adjacent to a lane of cones - it cannot yet do unconfirmed lane changes, although can take off ramps, so is unable to do the lane change automatically.

Did the whole lane move over, or did the inside lane close? If former, I have seen the car treat cones as a lane marking and follow them, but would not trust it in all circumstances. If the lane just closed down, then I think AP was doing its job of keeping you in lane although it is unclear what other safety alerts/actions this scenario would have triggered. Certainly I would have thought that a merge was outside basic AP functionality and a scenario where driver takes control.

Would be interesting to see how it would have behaved in a FSD equipped car.

I agree completely, I’m sure it is outside basic AP functionality. The left lane was coned off for roadworks, so all traffic was being merged into the outside lane. I was only slightly surprised by the behaviour because, like you say, on many occasions before AP has coped with it admirably, and simply followed the cones or barrier as the lane moves over. Luckily the road was very quiet, with little traffic. If it had been busier I would have taken manual control anyway.

 

[JCremonini]

On the M4 into London they are building extra lanes, on each side, which are currently behind solid barriers. There are a large number of random cones placed in the area and my M3 visualises them even though it’s impossible to drive anywhere near them.

Pedestrians are the best.

 

[Talllbloke]

Driving in Sainsbury’s car park yesterday I was surprised to see pedestrians visualised on screen. Not noticed that before, maybe it was in v40?

 

[AnRev]

Driving in Sainsbury’s car park yesterday I was surprised to see pedestrians visualised on screen. Not noticed that before, maybe it was in v40?

Mine saw a jogger! It flashed the alert (red collision warning ) too. I was not near him particularly but he was jogging ahead on same side of road.

 

[J1mbo]

The car seems to be able to spot cones from far and wide. They tend to dance around a lot on the display and sometimes the car shows a cone but none of the traffic around it.

Makes you wonder if the cones are being detected by a different NN to the vehicles & pedestrians.

 

[MrBadger]

Today I visited the superchargers at Fleet Services for first time since this thread was updated.

When finished charging, stall 1B, I was just about to pull away when I noticed that the adjacent supercharger had been visualised...



... as a wheelie bin with cone beside

Unfortunately I didn't stop and take a screenshot.

 

[CMc1]

Today I visited the superchargers at Fleet Services for first time since this thread was updated.

When finished charging, stall 1B, I was just about to pull away when I noticed that the adjacent supercharger had been visualised...



... as a wheelie bin with cone beside

Unfortunately I didn't stop and take a screenshot.

You’d definitely think they would add SuCs to the database as a priority before bins.

 

[AndrewGR]

I was amused to see that small food bins are visualised as full size wheelie bins. As others have said, it is perhaps surprising that parked cars (and skips) are not visualised when similarly placed bins and cones are.

 

[Mr H]

We have Metrolink trams in Manchester - one of them shows as a coach that won't go away until the whole tram has passed - especially funny if it's a 4 car tram

 

[Durzel]

Does anyone notice a slight lag with the visualisations? Is that intentional? When cars drive past me they’re probably 1 second ahead of the visualisation, rather than “real time”

I’ve not tested the latest firmware to see if it makes any difference.

 

[Dilly]

There is still a time lag.
Probably due to the system actively looking for anything that might be a traffic cone!

 

[NewbieT]

If you watch the marked up Tesla demo/recruitment vids you see that object identification and markup is done in real-time. It has to be otherwise there’s no chance of FSD ever happening. True, these vids aren’t necessarily marked by an actual FSD computer but there’s no reason why not.

Imagine processing every frame for each camera 30 times a second, something like: Identify objects, calculate their location, direction and speed, compare this with last known location. Predict what’s going to happen next, decide what action in controlling the car to take. That’s the FSD computer priority tasks.

There’s probably a lag in spatial data being passed out of the main processing engine. Like a frame rate I’d guess they push out a data cut every X cycles because of the overhead this creates in doing it more frequently (possibly using a slower bus). You don’t want to slow the main processing engine down with a data write-out delay just to get some smoother pictures on the display. So straight off the spacial data could be say 1/4 second behind. Then further time in data transfer and rendering the visualisations (not sure if this may even be done by the ‘iPad’ rather than the MCU).

Probably something to live with. I think it has got better since the first visualisations.

I wouldn’t worry too much the car isn’t visualising skips it doesn’t mean it can’t see or identify/recognise them, it’s just not set to display them. Although Tesla might not have gotten round to telling the system how skips behave (static object:: don’t drive into one).