EVNow
Well-Known Member
Whats notable is the "early beta" comment. They know safety score is very immature at present.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Safety Score
- Thread starter ScottNY
- Start date
Whats notable is the "early beta" comment. They know safety score is very immature at present.
FSD has been out for a year now in thousands of employee cars, and 71 public ones.
What kind of "issue" are they going to identify in the first 24 hours, having released on a Friday at midnight?
I mean, the only real data collection you can do in that time period is how many airbags went off.
If they seriously are rolling out based on data collection, not PR, then they're going to need weeks between batches.
And what is up with these midnight releases, meaning the first people to ever go and use FSD will have to do in the dark? Is that really actuarially sound?
EVNow
Well-Known Member
I made some comments on this earlier. For any model to work - you need to cleanse the data. 6B or 6M won't make a diff if its not cleansed. They have to figure out the data that should be excluded (thats what the data scientists should be figuring out and telling the engineers) and cleanse the data. Currently its like they threw a couple of entry level engineers to come up with a good fit for the dirty data they have.
OK, so who is going to Tweet Elon and ask if the FSD beta will actually have a manual detailing where and how it is to be used? Or are they just going to drop an update on 1K people at midnight with release notes of "cold weather improvements" and let hem have at it?
EVNow
Well-Known Member
Not how it works. This is really the first public rollout they are doing (the earlier non-employee rollout was for PR purpose and they obviously took a big risk). You monitor how it goes before rolling out further. Thats the standard industry practice that thousands of tech companies follow with every release.
What I'm surprised by is Musk not saying they will monitor for x days before rolling out further. He gives no timeframe.
"Cleanse the data" is just another word for a better algorithm. The algorithm needs to know what data to ignore by itself, or it can't be automated, and an algo at this scale only works if it's automated.
The data scientists look at all the data, and find what elements and behaviors have the highest statistical relevance to crashes. But they have to do it on the raw data they have, because that's all the algorithm can run on. You can't "cleanse" false FCW's from the data if the car is actually going to have them when you deploy.
That, we fully agree upon, but with the additional caveat that they were told to specifically not include acceleration or speeding. This whole thing feels like it was just an excel polynomial fit, down to the math they have on the "Safety Score" page
EVNow
Well-Known Member
Of course you monitor it. My point was that it's been out for a while, which means any "issues" it has could take much longer to surface than 24 hours on a Saturday. So it's not surprising to me that they are not going to triple their initial fleet size by Monday morning. Wouldn't you want at least a week to monitor this group?
This is going to be a very, very slow rollout. They're going to find "something" they don't like, and pause rollouts because "10.3 has big changes, no point in continuing 10.2".
The PR will say it released.... While also not taking the risk of a wide rollout.
Nope. Look at my post history. I do safety critical aerospace designs (which isn't IT either). And the last thing I'm allowed to do is "cleanse" the data before I start analyzing data, particularly when I need to run an algorithm directly on that data. If the data in is noisy, and I want to use future data collected in the same way, I have to have the algorithm deal with that noise internally.
Tell me, what kind of cleansing are you expecting Tesla to do to the 6B miles of data before they develop an algorithm off it? Maybe we're just using the word differently. I just don't see how you can collect a bunch of data, edit out some things, make a formula, and then deploy that formula to the exact same data without those things edited out and claim there is any relation.
Definitely further refinements coming to (early beta) safety test score. It will be refined continuously until it is an extremely good predictor of crash probability.
I don't see how the safety score is any kind of predictor of crash probability unless there are crashes. If there are no crashes then is a safety score of 15 any worse than a safety score of 100?
They aren't. The car is not brought to a stop in a fixed distance from a given speed. Not even a little bit close. Regen applies a braking force that decreases with lowered speed because the kinetic energy available to convert is decreasing. I drive up and down hills constantly where I live. This becomes obvious instantly. As I said before, regen won't even slow you down on many hills as the braking force available to regen is less than the forward force gravity is applying to the car.
I think I see where you're mistaken now. The accelerometer sees .1 G on a certain slope whether or not you brake, once you reach terminal velocity. Braking adds to that .1 G What you're assuming is that the car will be accelerating unimpeded if the brakes aren't applied, it won't be. There's plenty of drag available to keep that from happening. At some speed, the rolling resistance combined with aerodynamic drag will equal the accelerating force from gravity. Depending on slope this happens anywhere from a very slow speed all the way up to 40 or 50 mph for a medium gentle slope. (Medium gentle is relative to where you live of course.) Steeper slopes will result in a higher terminal velocity of course. It's been my experience that a slope that results in a terminal velocity as low as 20 mph combined with normal regen from above that speed will exceed .3 G and cause a hard braking penalty.
Daniel in SD
(supervised)
EVNow
Well-Known Member
Simple stuff - like take into account road gradient when looking for hard braking. Don't count FCW in the garage.
Basically, when you are analyzing data to come up with a model - the cleaner your data is the better your model. There are various things statisticians do to cleanse the data. Don't take my word for it - you can read about it in practically any text book on data science (or even you tube lectures that Sanford others have posted).
Not sure I follow you. Your score IS a predictor of a crash. it's based on probability. Now, there are tons of people who claim to be super safe drivers and never had accidents in decades. But a single person can and often go against probablity (getting lucky and flipping heads 10x for example). Across the fleet though, the safety score might already be an excellent predictor of crashes. The question is how much better does Tesla want to make the score so that it's a bit more individualized. I think they have the capability to do this if they leverage their cameras and identify other behaviors that are unsafe. They have to do this if they want to disrupt the insurance industry.
scottf200
Well-Known Member
Daniel in SD
(supervised)
There were crashes in the 6 billion miles of data, that’s how they came up with the safety score. People who have a lower safety score are more likely to crash.
The obvious refinement to the safety score will be to figure out what metrics correlate to FSD Beta crashes which is different than driving manually. Also they may want to somehow evaluate how well people using FSD Beta monitor the system to see who to kick out of the beta.
Sounds like you do mostly highway driving? Then yea what you’re saying would make sense. You can set AP to 80mph and get terrible Wh/mi while racking up a 100 safety score.
My driving has mostly been city where I am accelerating and driving significantly slower to build gaps between myself and cars driving ahead of me
Why do they have to do it in the dark???? All the midnight (historically that means 0300 PDT ) release means is that the first opportunity to use Beta will be in the dark, at least in the west half of the country. No one will "have to" rush out and drive in the dark. I'm guessing that most will sleep until their normal wake up time. And even if some of us are excited and rush out and drive in the dark, so what? Certainly the early morning hours in the dark are the safest hours to drive. The biggest threat to a good driver is other drivers, and that threat is minimized at 0400. Even wildlife encounters are down in the early AM, the highest level of wildlife activity occurs in the evening twilight hours and slowly drops in the first few hours of dark.
EVNow said his/her decrease in speed was much less on a 10% grade at 35 mph. The model 3 has a max regen power of 75 kw, so 47 mph is the max speed for 0.2g of regen. That car should be able to generate 0.2 gs of regen at that speed. Therefore is wasn't power limiting that decreased the acceleration.
So tacc determines a target acceleration. If it's on a hill the feedback accelerataion is wrong, so it doesn't apply enough regen/brakes. Next time through the loop, it's closer than it thought it should be, so it has to ask for more acceleration, but again the feedback value is wrong. Eventually the PID backs out the correct amount acceleration to apply, but then the slope changes.
If it's got an inner loop for acceleration, it's going to have to either use the bad accelerometer data or approximate it by calculating delta v over delta t. Why not just have the correct acceleration available?
Similar threads
