Safety Score

[JackSparrowV]

Most of the scoring happens when FSD is not enabled. When FSD is enabled then you should only get dinged if you get forced out of AP/FSD due to inattentiveness. Some people are getting dinged when AP violates the safety score rules but other people aren't. We don't know why.

So opt out will stop the scoring? Or we forever gonna be scored?

 

[CharleyBC]

I'm admittedly wavering. I told my wife she can only be a passenger in the Tesla for 1 week (she'll DESTROY the hard braking score). Now it's 2 weeks(?) and this game is already old. Haven't told her it's another week. You guys might have less competition soon...

I've wondered about this. Is the Safety Score per car or per driver? It'd be easy enough to do the latter, given the distinct profiles.

 

[Knightshade]

Sure.

The way this whole subject came up was - OP speculated that they will use some hidden filters apart from the safety score. I'm just contending that, that is not likely (esp. with camera data). Using camera data in a way not explicitly stated will always get bad publicity, besides there is zero reason why Tesla wouldn't disclose it if they were planning to use the camera data in any other way.

But the camera images don't go anywhere-- it'd only be reporting "was driver paying attention or not"- which is something we know the car already checks- and something we know they explicitly use to kick people OUT of the beta program. So I don't think there'd by any "bad press" concerns since you consented to this with the button anyway and the camera is even mentioned in the release notes.

As to if they consider anything BUT the safety score-Tesla is.....not always great at making everything clear though.

It wouldn't surprise me at all if engagement counted.

Likewise it wouldn't surprise me at all if location counted--- either giving bonus points for areas they're overfitting to if they're aiming for "more drivers in places we know the SW is already safer" or giving bonus points for areas they're UNDERfitted if the goal is greater testing in areas they know it needs more help.... depending on their goals either might make sense.


Wouldn't shock me if either is NOT being counted of course--- but why would they explicitly push the 'cameras for all remaining cars' update at the same time as the button if they didn't intend to use it?

 

[AlanSubie4Life]

but my overall unsafe following across all days INCREASED to 14.6%

That is weird. Maybe something to do with AP, or maybe something else. But as you say, does not matter. The numbers on the main Safety Score page are definitively not used in the calculation of your overall score.

 

[CharleyBC]

Ya know, our cars could more interactively train us to be safer drivers (according to Tesla's definition of "safe," that is). Let us turn on an optional display that shows the component metrics of the Safety Score in real time. For example, while cornering, I could see that was 2.7g. Oops, that corner was 5.1g. Stuff like that. It would not only remove some of the mystery of why our scores are moving up or down, but also give us real-time feedback so we could be safer. We already have real-time feedback of speed and power use.

 

[fiveten]

That is weird. Maybe something to do with AP, or maybe something else. But as you say, does not matter. The numbers on the main Safety Score page are definitively not used in the calculation of your overall score.

It seems like using those numbers would have been a better way to measure your overall safety score. Not sure why they just don't use those numbers

 

[brkaus]

This is interesting. I wonder why some people get dinged on AP and others do not. Is everyone using the same 2021.32.22 firmware? Maybe there's a difference between radar and no-radar cars. Maybe some of the autopilot or driving settings affect if you get dinged on AP or not. Someone who gets dinged a lot on AP should compare settings, firmware, etc with FalconFour.

I think k it’s getting into and out of AP at bad times causes the dings. I also feel AP is masking some things - like I had a bad day of yellow lights. AP trips scored much higher.
I’ve pretty much decided anytime the car leaves the garage the score only goes down. So I drive my truck now.

 

[HighZ]

Either its already a part of safety score (like above) or its not. If its not part of safety score already - it won't be used secretly by Tesla to filter.

When Elon says that any issues will be masked while on AP being used appropriately, the "appropriately" could be only when the driver's attention is on the road.

So if the camera detects a 'driver not paying attention', any issue that occurs while on AP would not be masked. That might explain why some events seem to be masked while others are not.

 

[FalconFour]

Following distance greater than 3 seconds doesn't impact the unsafe following score, so ignore that.

If you follow less than 3 seconds distance, that is your total "following time". If you then follow less than 1 second distance, that is the "unsafe" following time.

unsafe following score = time following less than 1s divided by time following less than 3s

So if you followed a car (less than 3s stopping distance) for 10s in total, but of those 10 seconds 6 seconds were within 1 second stopping distance then your score with be 6/10 = 60%

Following at greater than 3 seconds doesn't help because it doesn't count in the denominator.

If someone cuts you off closer than 1 second in distance and you immediately back off past 3 second follow distance you will have a high unsafe following score.

This, and this:

Yeah this confused me at first too. First, for this metric to count, you have to be following someone with 3 seconds of headway or less. If the car is further away from that, this metric is not even measured.

Once you are within 3 seconds of headway, the denominator starts counting time.

If during this period of time you get closer to the car where the headway is below 1 second, you generate the penalty numerator (also units of time).

So the final score is that fraction.

The more time you follow someone between 1 and 3 seconds, the larger your denominator and the less impact this penalty has.

have both proven to be game-changers. Today, as a test, I kinda flipped my typical behavior around: I used AP when there were no cars ahead, and I drove "by foot" the rest of the time. Guess what? My "unsafe following distance" overall score dropped from 38% to 5.6% as if by magic. (Because AP usage seems to mask everything - it also masks all the miles spent "safely following" using AP!) Still scored a perfect 100, but this time, just by "counting Mississippis". So lol, now there's another level of playing the game to go through I wonder if I can get my 99 up to 100 just by goosing the following score like this! It's really my only mark on the score to deal with (no FCWs, 0.2% on braking, 0.7% on turning).

Thanks to both of you for rephrasing Tesla's crazy wording. Now I totally get it.

 

[EVNow]

When Elon says that any issues will be masked while on AP being used appropriately, the "appropriately" could be only when the driver's attention is on the road.

So if the camera detects a 'driver not paying attention', any issue that occurs while on AP would not be masked. That might explain why some events seem to be masked while others are not.

That is highly speculative ...

Actually, we have no reports of the camera doing anything for non-fsd beta. First thing where you will see camera starting to get used is clear signals to put your hands on the wheel (or better still pay attention ! message). Not some sly way hidden behind "appropriate". Remember the idea is not to "deceive" people - its to warn them when they are not paying attention.

 

[HighZ]

Has anyone checked if a Smart Summon is counted as a trip?

 

[HighZ]

I h

That is highly speculative ...

Actually, we have no reports of the camera doing anything for non-fsd beta. First thing where you will see camera starting to get used is clear signals to put your hands on the wheel (or better still pay attention ! message). Not some sly way hidden behind "appropriate". Remember the idea is not to "deceive" people - its to warn them when they are not paying attention.

I agree, but we are all speculating right now. I was just trying to understand why some events are masked and others are not for some people. I had a FCW while on AP on a country road (caused by shadows across the road), but it was not counted.

Regarding the camera, the opt-in button did say that the camera would be used to detect attention. I have already gotten the "pay attention" alert if I am looking away for too long, maybe 10 seconds. And Tesla has never said what "using AP appropriately" even means. If they don't explain it, it already seems like a 'sly' remark. Do you know what it means (you can speculate)?

 

[nvx1977]

This, and this:

have both proven to be game-changers. Today, as a test, I kinda flipped my typical behavior around: I used AP when there were no cars ahead, and I drove "by foot" the rest of the time. Guess what? My "unsafe following distance" overall score dropped from 38% to 5.6% as if by magic. (Because AP usage seems to mask everything - it also masks all the miles spent "safely following" using AP!) Still scored a perfect 100, but this time, just by "counting Mississippis". So lol, now there's another level of playing the game to go through I wonder if I can get my 99 up to 100 just by goosing the following score like this! It's really my only mark on the score to deal with (no FCWs, 0.2% on braking, 0.7% on turning).

Thanks to both of you for rephrasing Tesla's crazy wording. Now I totally get it.

Anything before today has been locked in. but moving forward, you can prevent unsafe following from ever being an issue again now you know what to do

 

[JWM3]

You purport to have a 100 score in your avatar image, and I assume you have earned that score by going slow. My experience does not match. I currently have a 100 score, but I am still often going 10+mph over the speed limit on local streets. I just try to blend my speed with everyone else. The only thing I've been doing differently is leaving a lot more space between me and the car in front, so I can absorb braking over more time.

While sure going slower will also reduce hard braking and hard cornering, it's not something you really need to sacrifice to maintain a good safety score.

That’s what I figured too. I don’t drive slow, just driving normal and leave more space for more time to braking and use AP on freeways. Got 99-100 last 3 days.

 

[srs5694]

I just don't see how the Safety Score alone will be the deciding factor in who officially gets entered into the expanded Beta group.

Why not? Think of it from Tesla's point of view, rather than a Tesla owner's point of view....

The FSD-on-city-streets feature needs more beta testers -- but not too many all at once. How do we select them? We don't want the guy who was arrested last week for taking a nap on Autopilot to use it (yet), or the gal who did the same but was not arrested. So we come up with some metric to select drivers who are attentive. The metric doesn't have to be perfect, but it should be more likely to exclude Mr. Sleepyhead than to include him, and to include Ms. Attentive rather than exclude her. Using existing monitoring metrics for the Tesla Insurance program makes sense. Those metrics have been studied a bit. They aren't perfect, but using them is better than not using them. Coming up with a better metric would require considerable R&D effort, but Elon wants to push this thing out ASAP, so we don't have time to do that. Add a way for applicants to see their scores to keep them happy, and perhaps encourage good driving, and voila!

I don't know that the above accurately represents Tesla's (corporate) thinking, but I suspect it's close. Most importantly, the safety score is for Tesla's benefit and convenience, not ours. It's not meant to be perfect, comprehensive, or fair; it's driven by statistics, to get a reasonable sample of safe drivers, not the absolute best of the best of the best (to quote "Men in Black").

Also, remember that this isn't about getting FSD features vs. not getting them; it's about when you get them. You've been driving all your life without FSD, and going another week without it will (almost certainly) not kill you. In fact, given that it's a brand-new technology, getting it before it's fully baked is incurring a risk, especially to pedestrians and bicyclists, so we all might want to re-think our eagerness to get it.

 

[Raiders]

Why not? Think of it from Tesla's point of view, rather than a Tesla owner's point of view....

The FSD-on-city-streets feature needs more beta testers -- but not too many all at once. How do we select them? We don't want the guy who was arrested last week for taking a nap on Autopilot to use it (yet), or the gal who did the same but was not arrested. So we come up with some metric to select drivers who are attentive. The metric doesn't have to be perfect, but it should be more likely to exclude Mr. Sleepyhead than to include him, and to include Ms. Attentive rather than exclude her. Using existing monitoring metrics for the Tesla Insurance program makes sense. Those metrics have been studied a bit. They aren't perfect, but using them is better than not using them. Coming up with a better metric would require considerable R&D effort, but Elon wants to push this thing out ASAP, so we don't have time to do that. Add a way for applicants to see their scores to keep them happy, and perhaps encourage good driving, and voila!

I don't know that the above accurately represents Tesla's (corporate) thinking, but I suspect it's close. Most importantly, the safety score is for Tesla's benefit and convenience, not ours. It's not meant to be perfect, comprehensive, or fair; it's driven by statistics, to get a reasonable sample of safe drivers, not the absolute best of the best of the best (to quote "Men in Black").

Also, remember that this isn't about getting FSD features vs. not getting them; it's about when you get them. You've been driving all your life without FSD, and going another week without it will (almost certainly) not kill you. In fact, given that it's a brand-new technology, getting it before it's fully baked is incurring a risk, especially to pedestrians and bicyclists, so we all might want to re-think our eagerness to get it.

Great point and agree! I just would imagine that for a FSD Beta they would want people who actually use the current form of Autopilot software rather than only people who manually drive within the safety parameters. I'm totally speculating as are many others in this thread. I happen to love driving my car and if getting access to the FSD takes longer, I'm fine with that. However, I'm still going to "grandpa" my car for the next week and a half to hopefully hit a reputable safety score to get included.

 

[nvx1977]

I kinda remember from somewhere that pedestrians and cyclists, etc are considered soft obstacles and that the system is designed to prioritize collision avoidance on those objects over the hard objects. Of course the current state of FSD beta is not pedestrian-proof, but just pointing out that I like the prioritization there.

 

[scottf200]

I guess I have to decide what to do now…I have a score of "100" again (not really 100, but whatever...)

This score is an example of how the main score is not using the values on the same main score page; this set of values results in a score around 99.2. Instead, the mileage-weighted average score is used (as documented by Tesla). Furthermore, it looks like they are rounding the scores to the nearest integer before doing the mileage weighting (a big deal!):
My daily scores:
105 miles, 99 (98.51)
10 miles, 99 (99.05)
32 miles, 100 (100)
51 miles, 100 (99.63)
32 miles, 100 (100)

So, the formula is: ROUND( SUMOVERDAYS(DailyMileage*ROUND(DailyScore))/ TotalMileage )

There's no other way to get the result. The proof is: here are the various ways to do it, with my data:
<snip>

So basically I'm at 99.2 in reality, and the app is calculating 99.5 for the weighted average (exactly), and is rounding the display to 100. Got really lucky on that first day that I ended up with 99 rather than 98 - that would have really hurt (and in fact it does hurt my actual score).

If anyone sees an error here or some inconsistency, do let me know. I think the calculations are right.

Clever sheet you created. I did a basic one today for some data in an earlier post. I missed your post in this thread as there are so many. I replicated your number with my simple formula.

 

[AlanSubie4Life]

When Elon says that any issues will be masked while on AP being used appropriately, the "appropriately" could be only when the driver's attention is on the road.

So if the camera detects a 'driver not paying attention', any issue that occurs while on AP would not be masked. That might explain why some events seem to be masked while others are not.

I’d like this to be true - would be a great move on Tesla’s part - but so far I’ve seen zero evidence from anyone here that AP events are not masked. It seems like “whatever happens on AP stays on AP” (and does not enter the Safety Score).

I understand there have been a lot of claims, but people also are somewhat confused about how the scoring works, and I think every claim (at least on the freeway) can be adequately explained by the way the scoring works. It’s really pretty easy to get very high (bad) hard braking scores, cornering scores, and following scores, if you use AP a lot on the freeway - high (bad) scores for the components would be correlated with high AP use, even though no events on AP are counted (in fact, BECAUSE that is the case).

We’ll see though. People have to do careful experiments if they really want to figure this out. For example, put the car in park and exit the vehicle right before getting on the on-ramp, and after getting off the off-ramp. Etc. Pending good experiments, I’ll just go with the documentation. Suggestions for controlled experiments to determine the behavior on surface streets are provided earlier, too.

 

[EVNow]

And Tesla has never said what "using AP appropriately" even means. If they don't explain it, it already seems like a 'sly' remark. Do you know what it means (you can speculate)?

I've remarked on that "appropriate" not being defined or referenced elsewhere. It might be talking about the 2 year old intent of AP (divided highway). We don't know - but I highly doubt it has anything to do with the camera.

The reason I say that is because if its a new feature Musk / Tesla would be talking about it explicitly. Remember the idea is not just to actually get drivers who would pay attention - but also demonstrate to regulators and senators that they are doing a lot of things to filter out drivers. However flawed the safety rating is - it is in part to address the latter need.