AP2.0 Cameras: Capabilities and Limitations?

[verygreen]

So, can someone humor me on how the cars decide to upload snippets? Presumably this doesn't happen randomly. I suppose only when the car detects an unexpected event, the clip / context is uploaded.

So how do you define something as unexpected, unless you have an expectation? Who is creating the expectation. Calling it The Shadow mode could range from reasonable to charitable. But it's hard to say nothing of that sort exists.

I see @verygreen disputes the existence of the mode, but the question here is not one of existence, but it's capability. And to that, none of us has a clue.

Well, to reply you here as well:

Well, it DOES happen randomly. But not just that. Basically when Tesla needs more data - they upload "triggers" that match some criteria to a subse of cars. Typically those criteria range from "randomly with probability of x% capture something to "if we are about to hit something and it's time to trigger AEB - capture a snapshot". Triggers select what they want captured to from full video from front cars and 1fps vide from all cams to "just radar snapshot" or "just canbus snapshot". Triggers also have limits on how many snapshots they can create, ranging from 1 to a bigger number, often 10 to 20 for random events and 1 to 2 for nonrandom ones.

Those triggers go around and often have very limited duration, like 24 hours. The whole number of such campaigns at present are probably still under 100 (i saw caimapign #64 in November) and I have seen 4 of them (that should give you an idea about how many cars they hit with this at once).

The cars that don't receive a trigger don't do any snapshot other than unconditional ones. Early on unconditional snapshots were rich, say up until sometime in June every FCS wold generate a front cam video snapshot, but then they became very sparse. Now uncoditional ones pretty much include: component failure (something did not turn on/something crashed - cams not included unless it lead to a crash), the "rob-silent"/"rob-active" to detect false radar positives and the "car is crashing" snapshot when the car is detected to be in a crash.

So somebody on reddit unearthed an actual definition from Elon about the "shadow mode" that pretty much says "we'll make a snapshot when a condition is met" - and THIS does exist, but it's still a lot lesser scale than implied because not every car does get the conditions, the number of triggers is limited and the time the condition stays in place is limited. If the interesting condition happens without an active trigger - it's not reported.

But the problem is people put so much more into that definition, so before denying the existence of "shadow mode" I typically ask people what do they thing the "shadow mode" is, and oftentimes what they think about it does not actually exist.

 

[lunitiks]

^ Read this. Then, read it again. Then, read it once more. ^

 

[scottf200]

Well, to reply you here as well:

Well, it DOES happen randomly. But not just that. Basically when Tesla needs more data - they upload "triggers" that match some criteria to a subse of cars. Typically those criteria range from "randomly with probability of x% capture something to "if we are about to hit something and it's time to trigger AEB - capture a snapshot". Triggers select what they want captured to from full video from front cars and 1fps vide from all cams to "just radar snapshot" or "just canbus snapshot". Triggers also have limits on how many snapshots they can create, ranging from 1 to a bigger number, often 10 to 20 for random events and 1 to 2 for nonrandom ones.

Those triggers go around and often have very limited duration, like 24 hours. The whole number of such campaigns at present are probably still under 100 (i saw campaign #64 in November) and I have seen 4 of them (that should give you an idea about how many cars they hit with this at once).

The cars that don't receive a trigger don't do any snapshot other than unconditional ones. Early on unconditional snapshots were rich, say up until sometime in June every FCS wold generate a front cam video snapshot, but then they became very sparse. Now uncoditional ones pretty much include: component failure (something did not turn on/something crashed - cams not included unless it lead to a crash), the "rob-silent"/"rob-active" to detect false radar positives and the "car is crashing" snapshot when the car is detected to be in a crash.

It seems like it would be a massive amount of nearly "worthless" and repetitive data if they regularly took snapshots/images/videos from the 200,000 Tesla out there especially if 98% of the time they are doing the same routes. The triggers make sense when they need to train with more specific data but I think training with the repetitive data on the same routes wouldn't be worth it (storage, upload cost, processing cost, etc, etc). This seems too obvious to me so I must be missing something why people are surprised about this.

 

[verygreen]

It seems like it would be a massive amount of nearly "worthless" and repetitive data if they regularly took snapshots/images/videos from the 200,000 Tesla out there especially if 98% of the time they are doing the same routes. The triggers make sense when they need to train with more specific data but I think training with the repetitive data on the same routes wouldn't be worth it (storage, upload cost, processing cost, etc, etc). This seems too obvious to me so I must be missing something why people are surprised about this.

the surprise is in Tesla statements about billions of miles in shadow mode, I guess.

 

[croman]

Crazy that no one has mentioned this but I might also have the most miles driven with AP2. Anyways I was stopped with AP activated in a dense urban area and those donation collection people were around. Long story short, guy was 25 feet away but car threw up the HOLD and took it away when guy left view. On it's own.

This used to only be triggered by ultrasonics and would remain even after the triggering event disappeared. You had to tap the accelerator to clear the hold. It saw the guy and then saw he left. It was nearly instantaneous. Amazing. On 50.3.

 

[Pale_Rider]

Yeah I’ve never had it do that or been in a situation for it to, but have I noticed it generally slowing down for a pedestrian in a cross walk. Not always but about 75% of the time. I try not to freak them out too much and generally turn off AP before I scare them though lol

 

[croman]

I was stunned it even put up the hold based on vision alone and lost my sht when it took it off on it's own and started going.

 

[croman]

I was stunned it even put up the hold based on vision alone and lost my sht when it took it off on it's own and started going.

Tried it three more toldt and worked only once more time. Not sure why the other people weren't recognized as people. Rather disheartening as it's unreliable to say the least.

Also my radar failed in heavy snow. Not looking good for FSD in winter I guess. Wet snow. Parked outside instead of my garage so that might've contributed.

 

[scottf200]

Also my radar failed in heavy snow. Not looking good for FSD in winter I guess. Wet snow. Parked outside instead of my garage so that might've contributed.

Next time take a picture of how the car and all the sensors looked after you left it outside then thought all the sensors, camera, and radar would work flawlessly. There has to be some common sense and reasonable expectations here. Only use AP if the vehicles input sources are "clear"/"cleared" and conditions are reasonable, otherwise take utmost caution and don't complain if it doesn't work.

 

[croman]

Next time take a picture of how the car and all the sensors looked after you left it outside then thought all the sensors, camera, and radar would work flawlessly. There has to be some common sense and reasonable expectations here. Only use AP if the vehicles input sources are "clear"/"cleared" and conditions are reasonable, otherwise take utmost caution and don't complain if it doesn't work.

There was a dusting of snow. It was 34F. It quickly became a decent snowstorm but it wasn't a big deal when I came back to my car. In no way do I think it is acceptable to have such a fragile radar.

 

[Reciprocity]

Tried it three more toldt and worked only once more time. Not sure why the other people weren't recognized as people. Rather disheartening as it's unreliable to say the least.

Also my radar failed in heavy snow. Not looking good for FSD in winter I guess. Wet snow. Parked outside instead of my garage so that might've contributed.

My radar failed today with light snow build up. If because I have had pretty heavy show and I've build up and not had an issue.

Something wired happened the other day tough. Giant snow flakes dropping would show up as cars on AP and cause the car to break. Freaked me out. Thought the car lost its mind. The flakes where huge, like coffee saucers (I exaggerate, but they where very large). As soon as the heavy snow stopped, the problem went away.

The conspiracy theory has wondering if this was somehow detected by Tesla as a false positive and they are now limiting AP when heavy snow is the detected? Both issues where in different cars. Last month it was my x with the false positives and breaking and today was a model S on 18.4.1. which happens to be a patch or two past where the x was when it had the breaking.

It's probably not related but i have not had the radar issue before and there have been many heavy snow days in the last 2 years.

 

[CK_Stuggi]

The radar both on AP1 and AP2 is susceptible to icing/snow cover. Trouble is you can't really put heating writing in front of the radar to prevent icing, can you?

 

[croman]

The radar both on AP1 and AP2 is susceptible to icing/snow cover. Trouble is you can't really put heating writing in front of the radar to prevent icing, can you?

I thought Ap1 radar is heated but it's placement is lower and therefore nowm prone up icing.

 

[BigD0g]

I thought Ap1 radar is heated but it's placement is lower and therefore nowm prone up icing.

AP1 radar is heated.

 

[AnxietyRanger]

Both radars are heated AFAIK.

The problem is, AP2 and AP1 Model X radars are covered by a non-heated bumper.

There's a thread about it:

AP2 - Snow radar failure

 

[lunitiks]

Hello, rear radar :-)

 

[Kristoffer Helle]

Last winter, my AP1 radar stopped working one or two times. Ap2 several times.
This winter, AP1 has been disabled 50% off the time.... They changed something in the sw

 

[thomasmb]

Last winter, my AP1 radar stopped working one or two times.

Interesting, last winter my AP1 was disabled at least 50% of the time. AP2 only has been disabled twice since I got the car about a year ago.

 

[croman]

Interesting, last winter my AP1 was disabled at least 50% of the time. AP2 only has been disabled twice since I got the car about a year ago.

Wet sticky snow or slush are the radar killers. No matter how much powder dry snow comes down, it doesn't fret. Heavy rain spray can also disable AP 2 radar.

 

[bebbiXpress]

2 days ago I was driving during medium snow fall and neither TACC nor EAP could be engaged almost right from the start of the drive. There was no chance that snow or slug already blocked the sensor on the bumper so my guess is there was too much radar reflection by the falling snow or camera visibility was too low. I lean to the first as I was driving in worse condition (heavy rain) and EAP was still available.