Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Safety Score

This site may earn commission on affiliate links.
In other words, Elon saying "If driving behavior is good for 7 days, beta access will be granted" was not accurate- there will be many other factors included, and few people will get it.
Color me completely unsurprised that most people still think his tweets represent any useful commentary on reality.
It implies the next seven days, but the skeptic in me wonders if it mean the 7 days prior to whenever you land in the front of queue.
 
Link?

The Score is literally 115 - 22 x PCF, and PCF is rate per 1M miles. It's right there.

If your PCF is 0 (no accidents ever), your score is 115. It's that easy. Your actual PCF is 0.5 (1:2M miles)? Your score is 104. Fact.

I am doing the math by actual rate of collisions per million miles. Not going through their insane guestimates based on their random factors. This just proves their inputs don't ACTUALLY calculate a useful PCF, because a PCF of 0.48 (the actual fleet average) gives you a score of above 100. Fact.
Predicted Collision Frequency (PCF) = 0.682854x1.014495Forward Collision Warning per 1,000 Miles
x1.127294Hard Braking
x1.019630Aggresive Turning
x1.001444Unsafe Following Time
x1.317958Forced Autopilot Disengagement

PCF cannot be zero as you assumed.

 
A score of 100 may indicate that you are driving in an area/times where nothing interesting is happening. (No traffic, unexpected drivers getting in your way, pedestrians running in to the street, twisties, etc.) Tesla may not be interested in getting more uninteresting footage/testing. (It already does well in uninteresting/no traffic situations.)
Tesla‘s neural net training does love interesting situations, but I’m thinking reducing bad PR from collisions is the overriding concern at this time!
 
Driving on a side street near my house, I was forced to brake hard because a child started to run into the street from between 2 parked cars. If I had chosen to hit the child and continue, my score would've remained 100. Instead I'm at 98.

This is insanity.
By not hard braking you not only keep your score at 100, actually hitting pedestrians scores bonus points.

IIRC it was 10 for an adult, 20 for a teen and 50 for a kid or someone in a wheelchair.
 
PCF cannot be zero as you assumed.
Why not. Anything times zero is zero, and you can have zero of all of those events. 0.682854 x 0 = 0.

But again, I was using PCF in it's raw units. Predicted collisions per 1M miles.
Have an accident every 2M miles? Your PCF is 0.5
A PCF of 0.5 is 104% "safety score."
But the Tesla fleet average is 1:2.05M miles.....
Hence, the PCF calculated from the collected data is obviously not accurate, as the real world average of 1:2.05 is already off the chart.

Do you not agree that a score of 100% requires a PCF of 0.682854, which is one collision every 1.47M miles, which is lower than Tesla's own published numbers for their fleet average? Tesla purposefully caps everyone with a PCF of 0.682854 or greater to a score of 100, but this represents an actual collision frequency that is very low.

Shouldn't 50 be a median driver, not 104 if your actual goal is to give real drivers real feedback, not just feel good marketing?
 
Last edited:
  • Funny
Reactions: AlanSubie4Life
This is long, blame @kbM3 ;)
Really though, I'm glad I wrote this as my view on the situation changed as a result of it


So, what we know(ish): 0.3g of road deceleration to achive 6.7 MPH/s
The vehicle has accelerometers as part of the ABS/stability system. It also has at least one axis of gyroscopic sensing for stability control (yaw). It also has wheel speed sensors on each hub for ABS/ traction control.
Out of all of these, the wheel speed sensors are the most non-drifting and should be fairly accurate (its used used for the speedometer), so wheel counts over time gives speed and change is wheel counts over time over time give acceleration. This works well for averaging also. So one might think Tesla is using that with a 6.7 MPH/s acceleration limit (over the actual sample time) for the hard braking threshold.

However, acceleration is also available and would require less code to implement, so what happens if they used that?

Well, it depends on if they adjust their reference frame to back out gravity. On a flat surface while stopped, gravity is tangential to the road, Az (acceleration vertically) is 1G, Ay (lateral/turning) is 0, and Ax (forward/backward) is 0.
If stopped on a 30 percent downhill slope, Az =cos(30)=0.866, Ay=0, Ax=0.5 (gravity is pulling the car forward).

An accelerometer without orientation correction would view this situation as the car decelerating. Why? Because the force is pulling the vehicle forward which has the same effect on the accelerometer as the car accelerating backwards.

This is F=ma and Newton's laws at work. The accelerometer itself is a mass connected via spring like load cells to its housing. The mass's inertia resists changes in velocity and this produces a displacement and force measured by the load cells when the device is accelerated.

When the car accelerates, the rear of the packaging the accelerometer is housed in pushes against the sensing element in the forward direction. This force is read as positive acceleration (normally forward speed increasing). When braking/ decelerating the load element pushes against the forward edge of the packaging and a negative acceleration is reported.

Now, back to the incline. Gravity is pulling on the entire car, including the load cell in the accelerometer. Since the car is stationary, the accelerometer housing is also stationary and the forward edge pushes back on the load cell. Net result, it looks like negative acceleration. And, from the car's point of view, it is producing a -0.5G acceleration to keep the car motionless in the presence of the 0.5G forward acceleration due to gravity. (Note, that this is the result of Accel=Force/mass, with force generated being equal to the gravitational force and no work being accomplished).

One might think the system should have a three axis gyroscope and transform (rotate) and subtract out the gravitational acceleration to provide a correct picture of what the car is doing, and for some applications that might be true (like derived motor power calculations).
However, let's take a step back and look at what the system is trying to accomplish. The vehicle's phyiscal interaction with the world comes down to four areas of rubber interacting with the road. These forces do depend on the accelerations as reported. A stopped car is not providing acceleration via the motor, but the tires are providing an acceleration to prevent the car from moving. This acceleration corresponds to a force that the tire must exert on the road. If we tilt the road steeper and steeper, eventually there will be an angle where the force due to the vehicle's mass times acceleration due to gravity in the forward direction (gravity * sine of the angle) is greater than the force the tires can provide via the coefficient of friction times the force normal to the road surface (acceleration due to gravity time vehicle mass times the cosine of the angle).
At this point the car is uncontrollable, the circle of friction has been exited and we're in the skid zone. Not good from a control point of view. Now, use magic to stop the car and back off the slope just untill the tires can hold it again. How much braking authority do we have? None, were we to start rolling, there is no margin to stop the car again. It doesn't matter how little of an acceleration we are requesting, the system physically can't provide it.

So, if you want to make a system that stays in control of the vehicle and the vehicle is limited by grip do you care most about :
A: the change in vehicle speed
B: the acceleration due to the motors/ brakes
C: the accelerations/ forces at the tires?

I would say C
A and B are different ways to measure the same thing (until grip is lost)
C is the real world limit of the system.

Another example of why C is the critical constraint is banked curves. A car on a flat surface can only generate a certain amount of lateral acceleration (rate of turn) based largely on tire grip. However, by banking the curve such that the slope matches the resultant vector of gravity and the lateral acceleration, the car can controllably handle much high speeds/ lateral accelerations/ turn rates irrespective of grip. To an accelerometer in the vehicle, this maneuver, rather than looking like a turn with lateral acceleration, would instead look like gravity were increasing as the turn, slope, and gravity combine to form a vector tangential to the road surface.
Now imagine the opposite situation, the problematic reverse banked turn. In this situation the curve is banked but the opposite way, now we lose grip due to slope and the latteral acceleration makes us lose more. To the accelerometer, the bank appears as additional lateral acceleration and gravity appears decreased, exactly what the tires are dealing with.

Okay, I've been typing for over an hour almost two hours, so I'll try to wrap this up. If Tesla is dinging people when braking downhill at a lower level than when going uphill, they are not wrong in doing so. When headed downhill, there is less traction available for stopping due to gravity and the slope working against you. In the uphill case, they act to slow you, so less tire/ braking force is needed for an equivilent deceleration.
Further, if this is the case, the 0.3G figure is the driving parameter and the 6.7 MPH is an illustative value, but is only valid on a flat surface. The 0.3G is then really the coefficient of friction/ allowable tire force value the system uses as a do not exceed value.

So in the downhill case, one must start breaking earlier and with less force than if one were on a flat surface. Even though you may only be requesting 0.3G and the car is only slowing at 6.7 MPH, the tires are dealing with that plus the effect of the slope. The tires also having less of the car's mass to use toward friction, but that is not a huge factor. For a 10% grade (5.7 degree), forward acceleration due to gravity is 0.1G (same as the grade percentage) and normal force for the tires is only 0.5% less. Slowing the car at 0.3G in this situation requires 0.4G effective braking force (and that's what the accelerometer would report).
Final thought, think of regen, downshifting, or (worst case) riding the brakes down the side of the mountain. It requires continuous deceleration to keep the speed constant, and the tires are working even though the speed is not decreasing.

P.S. yes aerodynamics also figures in. In the mountains without brakes, your speed would stabilize once aerodynamic drag plus rolling resistance plus other losses equaled the force due to gravity with no dependence on tire friction. However, one cannot rely on calm air nor a head wind to stop the vehicle. Rather, to be conservative, one would plan for a tail wind. Also, aero force goes as the cube of speed, so less of a factor at lower velocities.
Great post. Did I ever page the right person. I definitely learned a lot.

But 0.3g with gravity included seems an unrealistically harsh limit on a downhill of any steepness. Tesla has to choose limits that are practical as well as safe. That’s why they allow 0.3g’s of braking and 0.4 g’s of turning and not using even smaller limits. Even those are unrealistically harsh and making Tesla drivers hated everywhere this week:)

One nit is that you said that tires impart an acceleration even at a stop on a decline, but I believe it should be tires impart the force necessary for 0 acceleration.
 
  • Like
Reactions: Sporty
I am doing the math by actual rate of collisions per million miles.
Remember the definition of accidents in their safety report, though.

Anything times zero is zero, and you can have zero of all of those events.
Anything (except 0 which is debated) to the 0th power is 1 though.

In reality even a very good driver who is perfect on all these metrics or any metric of safety has a non-zero PCF (someone can hit them in spite of their best defensive driving practices).

It’s an open question what the definition of a collision is in the data they used to create this model, since it is not consistent (as I understand it) with the published data. Might have been a newer portion of the fleet only, a different definition of a collision, etc.
 
Last edited:
  • Like
Reactions: gearchruncher
Why not. Anything times zero is zero, and you can have zero of all of those events. 0.682854 x 0 = 0.


PCF has a minimum value of 0.682854. The other formula which you seem to reference as indisputable (115 - 22 * PCF) was designed with the 0.682854 coefficient in mind. Because 22 * 0.682854 = 15, which then makes the max safety score a nice round 100. So if you feel that min PCF should be zero, then the other formula with the 22 is consequentially invalidated.
 
Well you highlight a problem with tying the notion of “safety score” to “usefulness for FSD beta”. It is safer to do hard stops whenever needed, so maybe it’s bad safety scoring to count it.
However,I think it is ok to include it in fsd eligibility- are you by choice or context in situations where, with human judgment, reactions and attention, need to slam the brakes? Maybe too harsh of an environment for the FSD beta, for now.
Hard braking does not require slamming on the breaks. I got one this morning driving the speed limit stopping for a yellow light. Your choice is to drive slower then the speed limit, or run the yellow.
 
  • Informative
Reactions: DCEV
I don’t think there’s that many people with 100 actually. The fleet wide average in the Safety Score app was 91 as of yesterday. If you roll out to people with a score of 99 or 100, that’d probably be <10%.
You also have people like @MP3Mike aiming for a score of 80-92 😆

Since just the other day my friends and I stopped pulling down the fleet average … so you can expect it to go up now. It could be a never ending cycle!
 
Hard braking does not require slamming on the breaks. I got one this morning driving the speed limit stopping for a yellow light. Your choice is to drive slower then the speed limit, or run the yellow.
Running yellow lights is one bad behavior this encourages. That has to be fixed eventually. Tesla does have the data, but it is more complex. The situation is especially bad if you’re at a low speed as it takes more time to go through the yellow light. That happened on my first drive. Running the yellow safely would have involved speeding up and turning rapidly, which also would have dinged my score.
 
Honestly I think the safety score should be based on the medium or mean number, whatever it is called; this throwing out the upper and lower extremes. As many have mentioned there are too many “normal” situations that are going to get you yellow or red flags in Safety Score.

That said it looks like Tesla is trying to find people with little or even no flags during the week to mitigate issues and bad press.

Which is interesting when you’ve watched over a 100 hours of the current and recent beta peoples drives. If they had Safety Score on to keep their beta it likely would have been removed from everyone that currently has it.
 
PCF has a minimum value of 0.682854. The other formula which you seem to reference as indisputable (115 - 22 * PCF) was designed with the 0.682854 coefficient in mind. Because 22 * 0.682854 = 15, which then makes the max safety score a nice round 100. So if you feel that min PCF should be zero, then the other formula with the 22 is consequentially invalidated.
Anything (except 0 which is debated) to the 0th power is 1 though.
But that isn't the formula. There is no multiplying by 0 in the Tesla Safety Score ever.


Thanks, I had not seen those as "raised to the powers" though, given the way they were written, and I was wrong. So I see the point there.

The other point stands. PCF is, by definition, rate of collisions per 1M miles.

A "perfect" score is 1:1.47M miles. Not great. Not terrible. All depends what they count as a "collision". They've sure avoided minor contact with all their previous safety reports. Why change now?

A score of 80% is about 1:0.630M Miles. A full 57% drop in rate of collisions changes your score by only 20%?

The reality is that any useful score would show most people at 50%, and only 1% of people would be in the 99-100%. Yet we have huge numbers at 100%. It's meant to pack everyone in that top 20% and make them feel good and superior, and only catch the very, very worst of drivers below 50%. Completely independent of how the PCF is calculated, the conversion from PCF to score is biased.
 
Hard braking does not require slamming on the breaks. I got one this morning driving the speed limit stopping for a yellow light. Your choice is to drive slower then the speed limit, or run the yellow.
I think I got one just because of high regen on an uphill. Still facepalming. Now gotta drive an extra 100mi to get the “per 1000 miles” down.
 
People arguing about the constructs in the safety score are really missing the point. What probably happened is that Elon got tired of reading all the whining about FSD and the button. He probably got mad and told the AI crew to get one of their interns to use regression analysis to come up with a safety score to create an illusion of selectivity. Tesla has accident data for its fleet so coming up with a significant collection of independent driving variables is not much of a challenge. We have no idea how big the error term is in the safety score model is, nor is it relevant. The reason is the Tesla legal department wanted some statistical "cover" when the NTSB and Senator Blumenthal come a-callin.

So those are the two reasons for this safety dance. Besides, my guess is that the range of scores around the fleet median is quite narrow, so Tesla can pretty much pick anyone who opted in as a new beta tester. It's more likely they will maximize regional diversity. They also may also avoid NYC and favor Kalamazoo. I suspect that the Sunbelt will also get preferential treatment to avoid having too many beta testers swerving around in the snow and ice. In any case, I'm blasé about the selection process whether I get annointed or not.
 
One nit is that you said that tires impart an acceleration even at a stop on a decline, but I believe it should be tires impart the force necessary for 0 acceleration.
A nit like this parenthetical?
And, from the car's point of view, it is producing a -0.5G acceleration to keep the car motionless in the presence of the 0.5G forward acceleration due to gravity. (Note, that this is the result of Accel=Force/mass, with force generated being equal to the gravitational force and no work being accomplished).

I pared down a longer tangent about acceleration vs accelerating which turned into the aero section.
Net acceleation is 0 when stopped, so the forces f1=-f2 balance, since f=ma, if m1=m2 then a1=-a2. Force due to gravity is cancelled and acceleration due to gravity is canceled.
But yeah, the tire acceleration (force) ends up being whatever cancels out the rest of the forces (accelerations) as opposed to an exact number like 0.5G