Safety Score

[powertoold]

I drove 100 miles today in varying traffic conditions in Bay Area. I got around 5% unsafe following and 0.3% aggressive turning.

I do think the Safety Score does a good job of representing my accident risk for the day.

I think that’s the problem people are having with the score. It doesn’t seem to represent your driving ability. Rather, it’s more representative of your accident risk in a given traffic locale (with your driving style mixed in).

 

[nvx1977]

I drove 100 miles today in varying traffic conditions in Bay Area. I got around 5% unsafe following and 0.3% aggressive turning.

I do think the Safety Score does a good job of representing my accident risk for the day.

I think that’s the problem people are having with the score. It doesn’t seem to represent your driving ability. Rather, it’s more representative of your accident risk in a given traffic locale (with your driving style mixed in).

which may be exactly what Tesla's looking for. safest people and safest PLACES to roll out initially.

 

[gearchruncher]

There is one speed on the road: the speed limit

You're proving just what a farce Tesla's "Safety score is."
Tesla already measures and knows if you are speeding. The car can even beep at you when you do it.
Yet, they do not use this in your safety score.
Yet they allow AP to be set above the speed limit
Yet, they allow FSD to be set above the speed limit.

But what Tesla does measure is braking, and turning, and FCW's (badly). And you really, really wish that Tesla could use this to set insurance scores?

This is exactly why we don't want private companies doing this- Tesla is motivated to not include speeding or acceleration in their rates because they actively advertise their cars as fast. Yet as you say, this is the one thing they should be doing.

Rolling through stop signs at 20+mph as if they said "yield" (and even then, failing to do so)

Oh look, another thing Tesla knows you are doing, yet doesn't include in the "Safety score"

Anything that can be spun to "it'll cost you freedoms and money! FREEDOMS AND MONEY!" will be. Best thing to do would be to find a way around it.

I thought you wanted them to use this because it would lower your costs, because you are a safe driver and other drivers are idiots.

It did, by creating an itemized list of things that insurers are allowed to use - that's inclusive, not exclusive - so new technologies and developments can't come along later to improve upon. It's a law that froze insurance in 1988, and that's it.

Sorry, nope, you're making up things. That un-changeable law? The phase "* An insurer shall only use a technological device to collect information for determining actual miles driven under the Second Mandatory Factor." was added in 2009. Not 1988. Look at the amendment list at the end of the law and tell me it was frozen in 1988. It's been amended about 50 times.

because nobody's there to stop them.

...And we're back to you wanting Tesla to be the "enforcer" of good driving, instead of the police.

 

[gearchruncher]

Rather, it’s more representative of your accident risk in a given traffic locale (with your driving style mixed in).

which may be exactly what Tesla's looking for. safest people and safest PLACES to roll out initially.

I agree with this, however, this is on Tesla for the way they describe the "safety score":

The Safety Score Beta is the first release to the Safety Score which is an assessment of your driving behavior based on five metrics called Safety Factors. These are combined to estimate the likelihood that your driving could result in a future collision.

The Safety Score Beta is intended to provide drivers transparency and feedback of their driving behaviors. The Safety Score is a value between 0 and 100, where a higher score indicates safer driving. Most drivers are expected to have a Safety Score of 80 or above.

Tesla is not making it transparent in any way that they believe *where* you drive has any impact. In fact, they make it clear that how much you drive doesn't even matter! If they are trying to estimate rate of collision on a calendar basis (which is all that matters for FSD "beta") then how could that be true?

Your daily Safety Score is not impacted by the number of miles or hours you drive.

 

[powertoold]

Tesla is not making it transparent in any way that they believe *where* you drive has any impact.

Pretty obvious to me, but it's not stated explicitly. Wouldn't that cause more drama if they mentioned that?

Insurance rates are based on driving locale, based on collision risk. Same thing with safety score.

 

[MarcG]

For FCW, it doesn't really matter how much more you drive in that same day (assuming you don't get more FCWs!) because the aggregate score looks at FCW over 1000 miles.

@AlanSubie4Life can you please expand on why you disagree with this? I don't care that you disagreed with my post, I'm much more interested in why you think the statement is wrong, as my calculations show this to be true (but maybe I'm doing something incorrectly, which is very possible).

 

[FalconFour]

You're proving just what a farce Tesla's "Safety score is."
Tesla already measures and knows if you are speeding. The car can even beep at you when you do it.
Yet, they do not use this in your safety score.
Yet they allow AP to be set above the speed limit
Yet, they allow FSD to be set above the speed limit.

But what Tesla does measure is braking, and turning, and FCW's (badly). And you really, really wish that Tesla could use this to set insurance scores?

This is exactly why we don't want private companies doing this- Tesla is motivated to not include speeding or acceleration in their rates because they actively advertise their cars as fast. Yet as you say, this is the one thing they should be doing.

Well, at least we can see eye to eye on that. Yeah, I was ... very surprised to find that Tesla didn't include speeding in the calculations. I see your point, though... there's things they could totally just willingly ignore, and factors they'll buff to just "seem safe". I just like the idea of having something quantifiable that someone can pay attention to - preferably something with grounding in road rules (like... blinker usage / lane-keeping, another frustratingly overlooked metric). I don't think Tesla should use THIS scoring game as an insurance metric, but that's why I'm so loudly saying that it's not... mostly as defense that "this system is crap!" to which I'm saying "I know it's crap, that's why it's not really an insurance test, lol".

Sorry, nope, you're making up things. That un-changeable law? The phase "* An insurer shall only use a technological device to collect information for determining actual miles driven under the Second Mandatory Factor." was added in 2009. Not 1988. Look at the amendment list at the end of the law and tell me it was frozen in 1988. It's been amended about 50 times.

ngl, I didn't know that. I'm piecing together what I do know from the sparse arguments I've seen online for making available to CA what other insurers have offered across the rest of the country for years. I mean, at this point, it's like... okay, it's well worn, it's used for over three-quarters of the country, but CA's just stuck in the sand. And to me it feels like it's stuck as yet another symptom of California's "asshole invasion" - the societal equivalent of "mods are asleep". A complete lack of any accountability is enabling A-holes to be a lot more ... expressive.

So, it really should be amended, then. I've actually sent an email along to the author of this article, asking what we can do about it: Modernize the way we price auto insurance – telematics is a sensible approach

Also, that section strictly relates to counting mileage by a telematics device... like I said, inclusive, not exclusive! It doesn't say what you can't use - it says what you CAN use - so it inherently blocks any future developments that the law isn't updated for.

 

[gearchruncher]

Pretty obvious to me, but it's not stated explicitly. Wouldn't that cause more drama if they mentioned that?

It's obvious to you that a algorithm, which explicitly does not contain location as part of it's data, and is described as "an assessment of your driving behavior" is judging you for both how you drive and where you drive?

If it's really doing that, Tesla should say it, and since when has Tesla cared about drama? Safety should be about transparency. Tesla should come right out and say that people that drive in cities are less safe, because their algorithm trained on 6B miles of data says so, and it cannot be wrong.

Insurance rates are based on driving locale, based on collision risk.

Yeah, which is why they ask you your zip code. Tesla specifically does not, and does not include this. It's intuited, but that is much, much different than using it directly.

 

[AlanSubie4Life]

@AlanSubie4Life can you please expand on why you disagree with this? I don't care that you disagreed with my post, I'm much more interested in why you think the statement is wrong, as my calculations show this to be true (but maybe I'm doing something incorrectly, which is very possible).

Here's my post on the topic: Safety Score

It most definitely matters how much more you drive the same day. You can drive more and make your score harder to fix (this would be the case if you're lucky enough to have an FCW in the first mile from your house and you turn around and come back), and you can drive even further and make it easier to fix (fewer total miles to end up back at a mileage-weighted average of 99.5) - this is generally what you should do if you've driven more than about 4-5 miles. This is because the aggregate score does NOT look at overall FCW rate per 1000 miles. It looks at the mileage-weighted average of the daily scores.

It's true that on the main score page, the "FCW per 1000 miles" value is likely independent of how many miles you drive that particular day. However, that value is not what is used to calculate your overall score.

The best plan of course is to not get any FCWs. They're very painful.

And yes, if you're fixing an FCW, you absolutely can't get another. Set FCW to Early and try to avoid triggering it. You also need to use that distance to work on getting all other detrimental factors, especially hard braking, to ~0.0%, for that day.

 

[powertoold]

If it's really doing that, Tesla should say it, and since when has Tesla cared about drama? Safety should be about transparency. Tesla should come right out and say that people that drive in cities are less safe.

It's implicit in Tesla's statement. Your collision likelihood is a combination of your driving and where you drive. Obviously.

 

[gearchruncher]

it's well worn, it's used for over three-quarters of the country, but CA's just stuck in the sand. And to me it feels like it's stuck as yet another symptom of California's "asshole invasion" - the societal equivalent of "mods are asleep". A complete lack of any accountability is enabling A-holes to be a lot more ... expressive.

The issue here is that we have no proof it works. Just because it's used doesn't mean it works, it just means insurers have been unchecked in other states. All those silly devices do is measure acceleration and braking and such as well. I had one for a while that dinged me just for getting off the throttle (manual car, lightweight, slowed down all by itself beyond the "braking threshold.") The same useless crap we have here. What we do know is that California doesn't have more accidents or deaths than other states. So what is that data collection doing? It's not making drivers safer. Do you really believe that drivers in other states are better than in CA, and this is driven because a small percentage of them have optional monitoring devices in their cars?

If you want to make a change, write Tesla and ask them for their data, and then go to work on it in a statistical way. PROVE that it's correlated. Prove that someone that doesn't brake more than 0.3G's gets into less accidents than those that brake more than that. Then go to your local representative, ask them for their time, and show them the data that this kind of thing works, and that you understand it didn't in the past but things have changed.

The irony here is that you want "Accountability" from "assholes" on the road, but you are asking for zero accountability from insurance companies. CA literally passed a law saying "insurers are accountable to the insurance commissioner" and you hate it, because you think it makes drivers unaccountable to anyone? You see how odd that looks?

I bet that if an insurer showed up with strong, big data driven, statistically relevant data, they could get that law changed. Insurers are big lobbyists. Maybe Tesla can be the first- but as you seem to agree, it will probably be by showing up with speed limit and stop sign running data, not braking or turning, because Tesla will be one of the first to ever be able to have that data. However, who the hell is going to sign up for insurance that dings you every time you go 3 MPH over?

 

[MarcG]

Here's my post on the topic: Safety Score

It most definitely matters how much more you drive the same day. You can drive more and make your score harder to fix (this would be the case if you're lucky enough to have an FCW in the first mile from your house and you turn around and come back), and you can drive even further and make it easier to fix (fewer total miles to end up back at a mileage-weighted average of 99.5). This is because the aggregate score does NOT look at FCW over 1000 miles. It looks at the mileage-weighted average of the daily scores.

It's true that on the main score page, the "FCW per 1000 miles" value is likely independent of how many miles you drive that particular day. However, that value is not what is used to calculate your overall score.

The best plan of course is to not get any FCWs. They're very painful.

Very interesting. Here's my experience:

Since joining the beta queue on 9/25 like most of us, I have had only one single FCW on 9/28 (reported here). That trip was something like 5 miles, so the FCW weighed something crazy (like 200) for that trip.

I took other trips that day, totaling 46 miles, so my FCW was 21.7 for the day (makes sense: 1 FCW x 1000 miles / 46 miles = 21.7 FCW per 1000 miles).

Since then, no FCW at all. If I look at my overall stats over 550 miles driven since 9/25, FCWs are sitting at 1.8, which again makes sense (1 FCW x 1000 miles / 550 = 1.8).

If I plug all the aggregate numbers into the equation for safety score, using 1.8 as the input for FCW, I land at the correct total score.

Perhaps this is same-same, but different? Either way, it looks like more miles driven without FCW = lower FCW value. Whether that's done on the same day or a different day, I'm not convinced it matters *for FCW* weighing purposes.

 

[powertoold]

People who are wondering why speeding isn't included? Well the NNs aren't reliable enough to read the signs, so it's difficult to be certain someone is speeding or not. Also, hard breaking, aggressive turning, and FCW will usually be worse if someone routinely speeds.

 

[FalconFour]

People who are wondering why speeding isn't included? Well the NNs aren't reliable enough to read the signs, so it's difficult to be certain someone is speeding or not. Also, hard breaking, aggressive turning, and FCW will usually be worse if someone routinely speeds.

This perfectly makes sense and is my experience too... I remember on a long trip on I-5 between SF Bay<->LA area, there would often be a "Trucks: 55 mph" sign just after the sign that said "Speed limit: 70". The car though the speed limit was 55 for about half that trip.

Tesla's map database is a mess... I just wish they'd work on crowdsourcing some cleanup, like how Waze does things.

 

[gearchruncher]

People who are wondering why speeding isn't included? Well the NNs aren't reliable enough to read the signs, so it's difficult to be certain someone is speeding or not. Also, hard breaking, aggressive turning, and FCW will usually be worse if someone routinely speeds.

Argument flow:
NN's are not good enough to read signs for a safety score. Just good enough to display on the screen and limit autopilot behaviors.

So let's use FCW's as a backup. They're perfect and not full of false positives......

Oh, well then braking and turning are a good proxy, even though a hill or yellow light will trigger braking and a 8 MPH turn on a green turn arrow will trigger turning. But not acceleration, oh no. That's not relevant to speeding.

Then let's use all of this to decide who gets the "mind blowing" "FSD beta" which relies completely on visual neural nets to drive the car around. Yet it can't read a speed limit sign accurately, which is totally cool. Good thing it doesn't need to see stop signs or anything to drive itself around.

Come on man. You can't run around claiming FSD is amazing and then say that Tesla can't even get speeds right well enough to use for some sort of made up "safety score". Just like it's obvious to you that the "safety score" also measures where you drive, it's obvious to me that the safety score is specifically designed to ignore things that Tesla knows would annoy their owners quite a bit- acceleration and speeding.

 

[gearchruncher]

Tesla's map database is a mess... I just wish they'd work on crowdsourcing some cleanup, like how Waze does things.

Why would they need to do this? Elon himself said maps are a bad idea and can't respond to changes. Real humans don't need a map to know the speed limit, and thus real Full Self Driving systems won't either.
Maybe we're not actually close to Full Self Driving at all if we can't read very specifically designed and regulated speed limit signs. Just a thought.

 

[AlanSubie4Life]

I took other trips that day, totaling 46 miles, so my FCW was 21.7 for the day (makes sense: 1 FCW x 1000 miles / 46 miles = 21.7 FCW per 1000 miles).

Since then, no FCW at all. If I look at my overall stats over 550 miles driven since 9/25, FCWs are sitting at 1.8, which again makes sense (1 FCW x 1000 miles / 550 = 1.8).

What I'm saying is that if you had driven ~156 miles that day instead, and ended with a score of 99 for that day (FCW = 6.4, all others at ~0.0%), you could have driven just about 312 miles (total, so 156 miles that day, combined with 156 perfect 100 miles on other days) to get back to 100.

Whereas with where you stopped, with a score of 94 (might have been lower with other demerits, but for the sake of argument...), you needed x = 46 *(99.5-94)/(100-99.5) = 506 miles more to drive at 100 to get back to 100. (So 46+506 = 552 total)

That's all I'm saying.

Another way to look at it: You could have driven 47.6 miles that day (48). That would have improved your score to 95 rather than 94 (FCW of 21 yields a score of 95).

Then you'd only need 428 more miles at 100 to get back to 100 (99.5). So 48+428 = 476 miles total. Drive 2 more miles that day, and save yourself 76 miles of perfect driving.

This "stepping" behavior is visible in those plots earlier. It's due to the score quantization that occurs.

If I plug all the aggregate numbers into the equation for safety score, using 1.8 as the input for FCW, I land at the correct total score.

This is a coincidence. For me, for some time, my hard braking score has been 0.3%, along with other demerits, resulting in a score of 99 if I calculate that way, but I'm a couple hundred perfect miles past "99.5" now in terms of the score calculation (meaning I'd need about 200 miles of 99 to knock me out of the 100 position). My score is 100.

The 1.8 on the main page is your true FCW rate, but it's not used in the formula.

Perhaps this is same-same, but different?

It ends up the same, but is different.

 

[MarcG]

What I'm saying is that if you had driven ~156 miles that day instead, and ended with a score of 99 for that day (FCW = 6.4, all others at ~0.0%), you could have driven just about 312 miles (total, so 156 miles that first day, followed by 156 perfect 100 miles on subsequent days) to get back to 100.

Whereas with where you stopped, with a score of 94 (might have been lower with other demerits, but for the sake of argument...), you needed x = 46 *(99.5-94)/(100-99.5) = 506 miles more to drive at 100 to get back to 100. (So 46+506 = 552 total)

That's all I'm saying.

Another way to look at it: You could have driven 47.6 miles that first day (48). That would have improved your score to 95 rather than 94 (FCW of 21 yields a score of 95).

Then you'd only need 428 more miles at 100 to get back to 100 (99.5). So 48+428 = 476 miles total. Drive 2 more miles that day, and save yourself 76 miles of perfect driving.

This "stepping" behavior is visible in those plots earlier. It's due to the score quantization that occurs.



This is a coincidence. For me, for some time, my hard braking score has been 0.3%, along with other demerits, resulting in a score of 99 if I calculate that way, but I'm a couple hundred perfect miles past "99.5" now in terms of the score calculation (meaning I'd need about 200 miles of 99 to knock me out of the 100 position). My score is 100.

The 1.8 on the main page is your true FCW rate, but it's not used in the formula.



It ends up the same, but is different.

Got it, thanks for the explanation. So to answer poor @yurmix 's question that started this whole conversion: it IS better to drive more on the same day (assuming no more FCWs) to increase your score for that day if you already had a FCW, and NOT stop driving for the day, right?

I'm curious why Tesla chose to use rounding that way instead of true numbers.. maybe it's to make it easier/more fair for someone not analyzing and dissecting numbers like we are

 

[S4WRXTTCS]

Here's my post on the topic: Safety Score

It most definitely matters how much more you drive the same day. You can drive more and make your score harder to fix (this would be the case if you're lucky enough to have an FCW in the first mile from your house and you turn around and come back), and you can drive even further and make it easier to fix (fewer total miles to end up back at a mileage-weighted average of 99.5). This is because the aggregate score does NOT look at FCW over 1000 miles. It looks at the mileage-weighted average of the daily scores.

It's true that on the main score page, the "FCW per 1000 miles" value is likely independent of how many miles you drive that particular day. However, that value is not what is used to calculate your overall score.

The best plan of course is to not get any FCWs. They're very painful.

Unfortunately there isn't any way to eliminate getting FCW's, but one can reduce the chances by reacting to things in a way an unexperienced driver would or a driver with a physical disability. Essentially you have to be more scared than the computer which freaks out about any little thing.

I have three main issues with the safety score:

It's not situationally aware. Any moderate to hard braking will be counted against you regardless of the cause. It doesn't matter if its a yellow light or a pedestrian in a parking lot that unexpectedly dashed in front of your car. They have all this intelligence to detect stop lights, pedestrians, etc and yet none of it is used to. As a result its more about WHERE you drive, and what time you drive than anything else.

It changes how people drive because instead of focusing on the act of driving, and doing what comes naturally they have to think about the safety score. Basically Tesla gamified trying to get the FSD beta, and this can lead to unsafe driving. The only reason I've done things like stopping for a yellow or a pedestrian is because I opted to weigh safety over my desire to get the FSD beta. Additionally its promoting people to drive more as its a way of improving ones score (assuming there isn't a FCW for the day). I'll probably need to go on some long drivers to bring my score up from 90 to above 98% without really any point to it except in hopes to get the FSD beta sooner.

It diminishes what a vehicle is to someone. I bought my P3D because it was capable of so many different things. The way I use it on a weekend on a remote mountain road has no reflection of how I drive it on my commute to work or my behavior when AP is on. I don't see a way of being able to go on a drive where nothing about the drive is counted. Heck its probably causing some spousal arguments as it doesn't seem to separate driver profiles. If they did that someone like me would simply create two profiles where one I drove like driving miss daisy, and the other I'd drive the way I'm used to after over 30 years of driving without any major incident.

 

[FalconFour]

Why would they need to do this? Elon himself said maps are a bad idea and can't respond to changes. Real humans don't need a map to know the speed limit, and thus real Full Self Driving systems won't either.
Maybe we're not actually close to Full Self Driving at all if we can't read very specifically designed and regulated speed limit signs. Just a thought.

The real world isn't exactly like Inception where roads just morph randomly on a daily basis. Hard map data is essential to FSD because not all cues are always visible or even well labeled. Fleet familiarity is what will make FSD safer than humans - but to what Elon is referring to, it doesn't need to be at the millimeter LIDAR level "HD mapping" that some companies claim is necessary.

Just having good rough data on a map, as they already have but is often incredibly inaccurate, is good enough.