Discussion of statistical analysis of vehicle fires as it relates to Model S

[Krugerrand]

everyone who has complained about data issues has made no effort to present anything alternative.

You're clearly an intelligent person, so I'm having a hard time understanding why you're stuck on this point. It's been made abundantly clear to you why they do not offer an alternative set of data, but let me try one more time for you: because it can't be accurately done, therefore results would be inaccurate, therefore not relevant

this is an investing thread. the smart money doesn't make investment decisions by throwing up their hands, saying "this sucks and can't be done."

It is, indeed, the investing thread and I'd contend that the *smart money* understands that endlessly massaging of numbers that can never reveal the truth is at the very best a big waste of time.

so you're entitled to your opinion about all the flaws and defects in the data, and you're also entitled to be parted from your precious capital by not attempting to come to any investible answer.

Yes, thank you for your concern for my precious capital and assuming I (or others, who currently disagree with your opinion) can't come to our own investible answer based on data that actually is not flawed or defective.

if you can't be objective, should you really be investing?

Who is that question posed to you? Nigel? Anyone who doesn't agree with your opinion? Or, yourself?

 

[BillHamp]

Those are two different questions. One is how prone the MS is to fire following collision and the second is how like you are to suffer injury in a MS fire after collision. Both are meaningful. We actually have better stats in the first case (weak as they may be) than in the second.

- - - Updated - - -

If lightening strikes on cars were as common as car accidents, then yes, the buyer should worry about lightening strikes.

 

[Mario Kadastik]

I was saying drawing concrete conclusions is not possible. I looked at your binomial and it's somewhat sensitive to the input low probability that you put there. And I've not dissected it enough to claim one or other. 2% probability is at least in my stats book still probable albeit I have to admit it's beyond the 95% CL that is usually quoted for exclusions. Then again 95% CL exclusions have excluded so many phenomena and are supposed to do it in 5% of the cases. That's why I'd say real evidence for a deviation is when we reach a significance of 3. One should compute not exclusion, but significance in this case.

Now I haven't read through the whole thread (moving way too fast for that and I've been busy finalizing my Model S delivery and payment) so am not going to attempt to compute it right now as I need to understand the input you're slamming in there. A naive and 5 minute check says that taking your latest probability for fire from ICEs and number of car years nets 7.3 expected fires hence observing 3 can't possibly be significant in either way, but again I need to understand the input first as it all comes down to the proper expectation function.

 

[BillHamp]

I was saying drawing concrete conclusions is not possible. I looked at your binomial and it's somewhat sensitive to the input low probability that you put there. And I've not dissected it enough to claim one or other. 2% probability is at least in my stats book still probable albeit I have to admit it's beyond the 95% CL that is usually quoted for exclusions. Then again 95% CL exclusions have excluded so many phenomena and are supposed to do it in 5% of the cases. That's why I'd say real evidence for a deviation is when we reach a significance of 3. One should compute not exclusion, but significance in this case.

Now I haven't read through the whole thread (moving way too fast for that and I've been busy finalizing my Model S delivery and payment) so am not going to attempt to compute it right now as I need to understand the input you're slamming in there. A naive and 5 minute check says that taking your latest probability for fire from ICEs and number of car years nets 7.3 expected fires hence observing 3 can't possibly be significant in either way, but again I need to understand the input first as it all comes down to the proper expectation function.

Mario, I would very much like to see your analysis in detail. I know you are busy, but it would be interesting and enlightening to see your perspective in its full glory. If you have the time, I'd be very much interested (even just from a pure probability/stats standpoint and not necessarily as it relates to this specific case)

 

[luvb2b]

Who is that question posed to you? Nigel? Anyone who doesn't agree with your opinion? Or, yourself?

i pose it to anyone (including elon) who can agree that teslas are almost surely safer than an ices when it comes to risk of electrical and mechanical fires (based on zero observations) and then disagree that there's something anomalous about the number of collision related fires.

both conclusions are reached by similar methods and data, yet to me it seems the tesla-positive conclusion is widely accepted and the tesla-negative conclusion is frequently derided.

regarding inaccuracy of the data, what is so inaccurate to render the conclusions faulty?

the key number being disputed is this: 4% of car fires are caused by collision.

so tell me, how far off is this number? is it off by a factor of 2? 4? 10? of course we know it can't be off by a factor of 25 (because that would put it over 100%). it's easy to say in a broad brush stroke "oh it's faulty." i say, fine, offer a correction. we know the number is between 0-100%. surely someone who disagrees could offer a sensible, conservative correction?

and also, i would ask why is the reported data off by so much?

1. is it that the number of collisions doesn't include all the "real collisions" with debris?
while i accept that, the collisions almost surely include any collisions with large objects.

2. is it that the mechanical and electrical fires include some collision fires?
those cases would be where a collision happened and much later caused a fire. but in the case of the tesla the fires happened immediately after the collision. most similar cases for ice's would be included.

3. do you realize that the fire data greatly overstates the fire risk for new vehicles? because old vehicles are much more likely to catch fire.

4. and the collision fire data includes some pre-collision fires (meaning a fire started, then the car crashed).

you see? if you think the data is wrong, just pick a figure for what percent of car fires are the result of collisions. decades of government research says 4%.

so say it's 8%. or 12%. all the conclusions still hold. that's how wide the margin of significance is here.

if you want to just consider fires from hitting debris, then let's pick a number for that.

and... the same applies to electrical and mechanical fires. tesla is so clearly safer than an ice in that regard, it's ridiculous!

i'm not trying to be snippy here - i'm just saying, if you want to keep telling me the data is flawed, fine. we have one key number we need to conduct the analysis: what percent of fires are caused by collisions?

you don't have to be spot on to conduct a sound analysis. but you do have to have a starting point!

 

[brianman]

Now I haven't read through the whole thread (moving way too fast for that and I've been busy finalizing my Model S delivery and payment) so ...

1. Breathe.
2. Finalize Model S delivery and payment.
3. Drink.
4. Eat.
5. Get a good night's sleep.
....
57. Read through threads like this.


Gratz, Mario!

 

[luvb2b]

A naive and 5 minute check says that taking your latest probability for fire from ICEs and number of car years nets 7.3 expected fires hence observing 3 can't possibly be significant in either way, but again I need to understand the input first as it all comes down to the proper expectation function.

yes, please do take the time to review carefully. the results are significant even at the 99% level.

regarding your statement above, the ice collision-fire per car-year probability i had was .0000392, multiply that by 10,820 car-years for model s = 0.42 predicted collision fires. i'm not sure where you got 7.3.

 

[imherkimer]

So, luvb2b, what is your ultimate purpose with this thread?
Seems to me you are trying mightily to discourage investment in TSLA.
You are flat out accusing people who are still favorable toward Tesla of being weak minded, and lacking in the "objectivity" that you believe you have perfected.
You even openly accused Elon Musk of deliberate deception (your truth vs his lies) in this regard. You seek to call into question the integrity of the car, the company, the CEO, and everyone who is favorable toward them.
You are working overtime here to convince the world that the car is statistically prone to fire and therefore ultimately unsafe. Furthermore your posts seem to relish the opportunity to castigate those who would disagree with you.
It seems to me that where you are really going with this is much further than what appears on the surface as a simple statistical analysis of crash data. You seem to further the implication that this fire issue (and the accompanying "lies" of the CEO) may be a problem which cannot be feasibly addressed, let alone "fixed" by Tesla Motors, even with a recall. If it could be determined that design of battery in the floor is statistically, fundamentally more prone to fire than gas cars, and therefore "unsafe," then this would require not just recall, but a complete redesign of the Model S. And since, at this point, Tesla has only one car on the market, a finding of fundamentally unsafe design of battery placement would mean be the end of Model S and therefore of Tesla Motors. You seem to imply a future catastrophic failure of this company, and its mission. You argue essentially that any "objective" person would realize this. It even seems that you hold a desire to see such a result. And this from someone who claims that, as Model S owner, the company has always been good to you.
Are you sure you have not established your short position? If you really believe what you write here then it would surprise me if you did not go all in on shorting this company. Your work here in this thread does not seem to be intended as constructive criticism, rather it seems intended as a warning to longs to get out while we can, an accusation that Tesla is fundamentally dishonest as a company. As such you are providing an arsenal of negative arguments for shorts and anti-Tesla interests to spread at will. At a minimum, you seem to have a manifest intent to undermine the confidence of TSLA investors. ("If you can't be objective, should you really be investing.") And of course you just "proved" that those of us who are long TSLA are lacking in objectivity, otherwise we would not still invest in the company after reading your analysis. Right?

 

[stopcrazypp]

Those are two different questions. One is how prone the MS is to fire following collision and the second is how like you are to suffer injury in a MS fire after collision. Both are meaningful. We actually have better stats in the first case (weak as they may be) than in the second.

We actually do not have the first. We have the probability of collision fires. But (and this is quite subtle) we don't have the probability for "after a collision how likely will the Model S catch on fire". And for that we need to compare collision rates.

As for the whole MIT article, he did not really do any deep analysis. He only multiplied the numbers for the Model S by 4% (portion of fires caused by collision in ICE cars), and made the same assumption that the collision rate is the same as ICE cars (I already showed up thread, at least for debris related collisions, the Model S has 3x likelihood of the other vehicles just based on the 5 debris related Model S collisions that have been reported). And his conclusion is incorrect (that the stats show Model S is more prone to fire overall). His stats only show the Model S is more prone to collision fires, but not overall fires.

 

[Krugerrand]

...regarding inaccuracy of the data, what is so inaccurate to render the conclusions faulty?

I couldn't possibly explain it to you. The people before me didn't have the words for you and I surely do not. You've entered a state that does not allow you to receive information not in direct agreement with your conclusion. This is not unique to you, it happens to all of us in our lifetimes...some more than others.

Just look at your answer to my question (a simple question I might add that only required a one or two word answer, yet)... Was I not clear in my previous post/s about where I stand?

i'm not trying to be snippy here - i'm just saying, if you want to keep telling me the data is flawed, fine. we have one key number we need to conduct the analysis: what percent of fires are caused by collisions?

First, I could careless if someone gets snippy or not. I can't be offended because I don't give others the power to offend me. So, I'm perfectly fine with however I am answered, and I'll go on record (so that your post doesn't get moved) that I didn't find it snippy in the least. I find it passionate, among other things.

Since you seem to want a number or statistic or something of that nature from me, I'll tell you the only thing I will consider. The number of ICE collisions caused by running over a trailer hitch at highway speeds that resulted in a fire (and of course what percentage of injuries there were, percentage of fatalities, percentage of causing a multi-vehicle pile up etc...) added to the number of ICE collisions caused by running over a piece of truck bumper at highway speeds that created 25 tons of impaling force and caused a fire (and of course what percentage of injuries there where, percentage of fatalities, percentage of causing a multi-vehicle pile up etc.... Find those numbers (preferably from more than one source - because I know how people collecting data can make mistakes so I'd like corroborating numbers) and compare to the two Model S fires to your heart's content and I will be all ears.

you don't have to be spot on to conduct a sound analysis. but you do have to have a starting point!

I have a starting point. Pay attention when you drive. Don't tailgate. Don't follow vehicles with mattresses tied to the roof, or that have appliances hanging out of the trunk. Don't follow gravel trucks and others of that nature. Watch a good distance in front of you for debris on roads, for swerving vehicles etc... Always be aware of your surroundings and have an 'exit strategy' should something out of the ordinary happen. Don't let people sit in your blind spot. Don't sit in other people's blind spots. Don't pass on the right, etc...

That starting point has served me well for 30 years of driving, such that I have never been the cause of an accident, such that the handful of accidents I have been in I have been able to take action to prevent being hit head on, being t-boned etc.., and such that I've never run over any type of debris that might otherwise impale my car, cause extensive damage, start a fire, cause me to lose control and so on...

Frankly, I'm baffled that anyone would invest in any car manufacturer based on the type of statistical analysis you want to do with Tesla. Because surely with all the fires and recalls and such that all the other big names go through year after year, nobody in their right mind would invest in them, let alone buy their vehicles. Yeah, I'm being sarcastic.

 

[Mario Kadastik]

yes, please do take the time to review carefully. the results are significant even at the 99% level.

regarding your statement above, the ice collision-fire per car-year probability i had was .0000392, multiply that by 10,820 car-years for model s = 0.42 predicted collision fires. i'm not sure where you got 7.3.

I'll look into it a bit more, but a simple Poisson probability for an average expected event occurrence of 0.42 gives 5.8% probability for 2 fires and 6.7% probability for >= 2 fires. I'm assuming the collision stats you took were for US therefore it would not be statistically quite valid to include the Mexican fire. Which means that we're around the 93% region so can't exclude at 95% confidence level nor can we claim it significant because we're still far from 3 sigma. Even if we include the third fire and I'm not 100% sure that'd be quite valid with the mean expectation as it may well be the crash ratio with fire is far higher in Mexico, then we get that the probability of >=3 is 0.9% and < 3 is 99.1%. The 3-sigma level is 99.7% and we have so far not accounted for any uncertainties therefore the real significance is smaller for sure so can't even claim 100% that we could really exclude at 95% CL as the uncertainties might very well shift the outcome. If I get time I'll try to add some uncertainties to the estimates and run it through the Higgs exclusion and significance estimator tools to find some more precise numbers.

 

[bonnie]

Guys, this whole conversation is completely off the rails at this point. Reading through the thread, it comes off as if folks just want to demonstrate their knowledge of statistics ... and I'm sure that's not the intent. But that's how it comes off. What started as a well-intentioned discussion has (from where I sit) become somewhat absurd.

We can't even agree that there is enough data (and in my line of work, there is not) to form any valid conclusions with any level of certainty.

If we are going to turn this into a discussion on the application of statistics, perhaps we would be better served to not talk about the specific incidents (therefore giving fodder to the countless number of media/shorts looking for a soundbite) and just changing it to something else. (Not a direction from a moderator. Just my two cents as an individual.)

 

[FreeOfPge]

All the statistical analysis on TMC is about to start a new type of Tesla fire inside my head! We went 15 months from the first retail deliveries on 6/22/2012, then we had 2 fires in 5 weeks. The current rate of fires as of Sept 30, 2013 was ZERO (road debris) fires every 15 months. On October 1, 2013 the first fire occurred and that rate changed to One fire every 15 months. Then on November 6, 2013 that rate changed again to Two fires every 16 months. Today is 11/22/2013, it has been 17 months since retail deliveries began and there have been two fires in a 17 month period or one fire every 8.5 months. At the current rate, without all the analytical noise, the 100th fire will happen 69 years from today.

 

[BillHamp]

We actually do not have the first. We have the probability of collision fires. But (and this is quite subtle) we don't have the probability for "after a collision how likely will the Model S catch on fire". And for that we need to compare collision rates.

As for the whole MIT article, he did not really do any deep analysis. He only multiplied the numbers for the Model S by 4% (portion of fires caused by collision in ICE cars), and made the same assumption that the collision rate is the same as ICE cars (I already showed up thread, at least for debris related collisions, the Model S has 3x likelihood of the other vehicles just based on the 5 debris related Model S collisions that have been reported). And his conclusion is incorrect (that the stats show Model S is more prone to fire overall). His stats only show the Model S is more prone to collision fires, but not overall fires.

I disagree, but I'm tired of saying the same things, so I'll leave it at that. I look forward to an analysis from Mario as I think it will help elucidate why there are two views of signficance regarding these events.. So far, analyses that control for as many variables as possible seem to disfavor what Elon is saying. Hopefully the NHTSA will provide a thorough review soon.

 

[Krugerrand]

I would like to add for luvb2b that from an investment standpoint you're looking at the wrong end of things and asking the wrong questions. Whatever the statistics you come up with, they are not the thing that one should be basing their objective investing decisions on. The more relevant questions for short term and long term investing are ones such as:

What happens if the NHTSA concludes there's not a thing wrong with the Model S - carry on at will, Tesla?
What happens if the NHTSA concludes there's not a thing wrong with the Model S and reaffirm via a new set of testing that it's still safer than anything else out there?
What happens if the NHTSA concludes the Model S needs a recall and design overhaul?
What happens if the NHTSA concludes the Model S needs a recall for a retrofit solution that Tesla and the NHTSA have already discussed and agreed upon? What happens based on a relatively expensive retrofit vs a relatively inexpensive retrofit?

Those are your investment questions, not the statistics.

 

[luvb2b]

...I'll tell you the only thing I will consider. The number of ICE collisions caused by running over a trailer hitch at highway speeds that resulted in a fire (and of course what percentage of injuries there were, percentage of fatalities, percentage of causing a multi-vehicle pile up etc...) added to the number of ICE collisions caused by running over a piece of truck bumper at highway speeds that created 25 tons of impaling force and caused a fire (and of course what percentage of injuries there where, percentage of fatalities, percentage of causing a multi-vehicle pile up etc.... Find those numbers (preferably from more than one source - because I know how people collecting data can make mistakes so I'd like corroborating numbers) and compare to the two Model S fires to your heart's content and I will be all ears.

maybe we're getting somewhere.

i am sure that you know the exact things you're asking about won't be addressed anywhere. i'll present to you an example source, see if you find it acceptable.

here is a study by aaa that another user posted.
https://www.aaafoundation.org/sites/default/files/VRRD.pdf

this particular study looked at many different databases in many different ways to find a comprehensive view of how debris can cause crashes. the summary of the report is on pages 9-10 "When cargo or a vehicle part dislodges from a moving vehicle and falls onto the road, it becomes a serious hazard for road users. Even a small item may be dangerous when it is discharged at highway speed prompting erratic avoidance maneuvers, and causing a crash. Vehicle-related road debris (VRRD) is material—vehicle parts or cargo—that has been unintentionally discharged from a vehicle onto the roadway. This study examines the safety impact of VRRD on North American roads." so it addresses specifically what you are describing, objects like a trailer hitch and a truck bumper that came off another vehicle.

their result is "A close examination of several crash databases, in some cases including a review of individual crash reports, was conducted to estimate VRRD crash frequency and severity. The crash data assembled and analyzed in this study consistently indicate that VRRD is a causative factor in less than 1 percent of all crashes. The data sets that were given the most thorough review provide the best estimate of the VRRD crash rate, about 0.4 percent of all crashes. All of the data sets suggest that the severity of VRRD crashes is less than that of other crashes; moreover, this finding is supported by a lower fatal VRRD crash rate (0.2 percent) yielded by data from the Fatality Analysis Reporting System."

if you look on page 31 of the study, you'll see they include all types of crashes where some debris was involved. it includes hitting the debris, or hitting other cars while avoiding debris.

would you accept the conclusion from this source, that 0.4% of all crashes are caused by vehicle-related road debris (which includes, but isn't limited to, tow hitches and bumpers falling off vehicles)?

 

[BillHamp]

Guys, this whole conversation is completely off the rails at this point. Reading through the thread, it comes off as if folks just want to demonstrate their knowledge of statistics ... and I'm sure that's not the intent. But that's how it comes off. What started as a well-intentioned discussion has (from where I sit) become somewhat absurd.

We can't even agree that there is enough data (and in my line of work, there is not) to form any valid conclusions with any level of certainty.

If we are going to turn this into a discussion on the application of statistics, perhaps we would be better served to not talk about the specific incidents (therefore giving fodder to the countless number of media/shorts looking for a soundbite) and just changing it to something else. (Not a direction from a moderator. Just my two cents as an individual.)

What is your line of work in which there not being enough data in such a situation? This thread was moved here specifically to discuss that statistics of the situation, so it seems entirely appropriate to being having a discussion in which people present statistics and argue about them. It is hardly "off the rails" if the discussion is exactly what the thread is about.

- - - Updated - - -

I would like to add for luvb2b that from an investment standpoint you're looking at the wrong end of things and asking the wrong questions. Whatever the statistics you come up with, they are not the thing that one should be basing their objective investing decisions on. The more relevant questions for short term and long term investing are ones such as:

What happens if the NHTSA concludes there's not a thing wrong with the Model S - carry on at will, Tesla?
What happens if the NHTSA concludes there's not a thing wrong with the Model S and reaffirm via a new set of testing that it's still safer than anything else out there?
What happens if the NHTSA concludes the Model S needs a recall and design overhaul?
What happens if the NHTSA concludes the Model S needs a recall for a retrofit solution that Tesla and the NHTSA have already discussed and agreed upon? What happens based on a relatively expensive retrofit vs a relatively inexpensive retrofit?

Those are your investment questions, not the statistics.

Those questions are answered by the stats to some extent. If luvb2b correctly predicts the significance of these events, then he'll know the NHTSA decision to a high degree of accuracy before they make it. That puts him in a great position in terms of investment as what the NHTSA says will likely have a huge impact.

 

[Krugerrand]

i am sure that you know the exact things you're asking about won't be addressed anywhere.

Naturally. But I'm an apples to apples kind of a gal. In fact, sometimes I'm a McIntosh to McIntosh kind of a gal.

would you accept the conclusion from this source, that 0.4% of all crashes are caused by vehicle-related road debris (which includes, but isn't limited to, tow hitches and bumpers falling off vehicles)?

No, but what I will accept is that it's a relatively low percentage of all crashes that are caused by vehicle-related road debris, the same way I'd accept that it's a relatively low percentage of collisions with road debris that cause vehicle fires, which I'd not have required a 148 page study to come to that conclusion. It's simply a matter of common sense, a little intelligence and life experience.

- - - Updated - - -

Those questions are answered by the stats to some extent. If luvb2b correctly predicts the significance of these events, then he'll know the NHTSA decision to a high degree of accuracy before they make it. That puts him in a great position in terms of investment as what the NHTSA says will likely have a huge impact.

The only way his stats predict the outcome of the NHTSA's decision is if the NHTSA uses his exact data, comes to the exact same conclusion AND that's how they run their organization ignoring all the other pertinent criteria of human safety as it relates to vehicles, as well as the level of risk we must all accept when driving a vehicle, etc... Certainly I could be mistaken, since I don't work for the NHTSA, but I suspect they don't simply rely on inaccurate statistics from a too small sample size.

 

[stopcrazypp]

here is a study by aaa that another user posted.
https://www.aaafoundation.org/sites/default/files/VRRD.pdf

This got buried in the thread, but I posted previously that this shows the Model S has 3x the likelihood of getting involved in a debris related collision (and that's before limiting to ones that did not involve collision from attempting to avoid debris).
http://www.teslamotorsclub.com/show...tes-to-Model-S?p=503812&viewfull=1#post503812

Summary:
25k debris related collision/year/190 million vehicles(car-years): 1/year/7600 vehicles
5 debris related collisions/year/25k Model S: 1/year/5000 Model S
5 debris related collisions/year/13k Model S(car-years): 1/year/2600 Model S

 

[NigelM]

to nigel - with all due respect as moderator - your first comment was that an analysis can't be done. i showed that it can be done, detailed two different generally accepted procedures on how to do it, and had results confirmed in posts by a phd, an md, and a couple other people on the board. the remainder of your comments added nothing to the discussion and seem kind of smart-alecky. if you don't know how to analyze this kind of data, i think you should just accept that and step aside.

Firstly, please stay civil, there's no need for personal comments. I can handle the analysis and a p***ing contest to see who's got the best qualifications is not going to add to the discussion.

You selectively quoted me previously but I specifically stated earlier that I wasn't questioning your methodology; I (and others) have questioned the validity of the basic assumptions and as long as you do not accept that view (I'll defend your right to a different point of view) there's no point in going back and forth with meaningless numbers. I'm assuming you're a smart guy and that I've just failed to get the point across so I'll give it one last try to show you what I mean:

• You stated up-thread "the models predict you'll see another collision-fire before the end of 2014q1". Now I'd like to be fair and say that you also gave ranges but this was your best prediction based upon your mathematical models.
• If there isn't a collision/fire in the next 18 weeks your analysis is completely off. The lack of a single event in a relatively short time period will skew the whole model. Alternatively, same goes if there is another single event next week.
• Taking a dataset of three accidents (an accident by definition is not predictable as it depends upon a random set of circumstances) and basing a specific prediction against general data is really lacking validity in my mind, regardless of whether your methodology is perfect or not.

I tried my best and you're welcome to the last word if you wish.