Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Discussion of statistical analysis of vehicle fires as it relates to Model S

This site may earn commission on affiliate links.
i think the first thing i realized is that if i could know that the odds of a model s collision were less than or equal to the odds of an average ice collision, then my test would err in favor of tesla.
However, from data given by another poster, at least for debris related accidents, the Model S seems 3x more likely to get into one based on the current reported incidents:
http://www.teslamotorsclub.com/show...tes-to-Model-S?p=503812&viewfull=1#post503812

i think it's reasonable to assume that the model s drivers are likely to be either more educated or have a white collar job vs. the average ice driver. for example, it's probably safe to say a lot fewer teenagers are driving teslas than ices.
I don't think looking at just one factor is enough, as there's way too many variables. For example, the Model S population is primarily male and young. It's also a low slung and quick sports sedan. Both of those are high risk in the insurance industry.
http://www.cnbc.com/id/100970641

however, another thing i learned along the way is that the nfpa statistics cannot distinguish between a pre-collision fire and post-collision fire in most cases.
That could skew either way. For example, what was pointed out previously (a fire reported as mechanical failure even though it was a debris collision).
 
Last edited:
I guess I'm late to the party :)

First off: Welcome back luvb2b! Great to have you back on the board. Second: Great attempt at an in-depth analyzis, one that is much better than Elon's ever was to begin with. Also thanks for engaging in further discussion and for a lot of great input from others in this thread.

My background is I'm working as a medical doctor and I'm working on a PhD right now and have over the years gotten a pretty good hang of statistics. I applaud you luvb2b for really going deep in to the source material from the and trying to make sense of the numbers. However, my way of looking at this would be (probably colored by my experience from using statistics in the medical field):

We have a few events that has spawned our interest - the Tesla fires. Our basic question that we are all posing, and I agree it's important exactly how we formulate it, is something like: Are Tesla cars more prone to fire when colliding [with road debris] than ICE cars. The actual events that we have are either 3 or 2 whether you decide to include the fire in Mexico or not. It is a fire after a collision so why not include it, however the concern raised that it happened outside of the US is correct in one way, but irrelevant in another way (since it's a Tesla no matter where in the world it was driven). The problem here is the small sample size, both with regards to number of incidents and with regards to the Tesla group as a whole no matter if you look at it from the point of number of cars ever made, car-years (number of cars times time in existence) or if you choose to look at miles driven or miles driven per year. Since the Tesla group - study group - sample size is so small it really doesn't help that the control group - "ICE cars" - is so huge: there will be a lot of innate statistical insecurity in the data, hence wide confidence intervals/high p-numbers.

The next and really biggest problem is how to define the control group and how to get good data to use for the control group. The way I see it this is a retrospective cohort study. One cohort is the Tesla groups with its few events, the other cohort we can try to define. The best definition IMO would be cars comparable to the Model S which would be cars made in 2012 and 2013, for personal transport, and excluding small trucks, large trucks, motorcycles etc. etc. To get an even better matched control group I guess you could go on to pick out makes and models that are even more comparable when it comes to performance, price, weight etc. This is what Tesla have been comparing themselves to before with regards to safety, pricing, performance etc. For example Mercedes S-class, BMW 7-series, Audi A7 etc. You will never find this data though so it's a lost cause. The NTHSA may have more detailed data but it's not publically available. Hence we have to do with a control group made up of a very mixed group of vehicles with large variation in age, type of vehicle, etc. etc. The fact that the data of the control group is extensive when it comes to number of incidents, miles driven, time of observation etc. doesn't really help much IMO since no matter how you look at it it will be an apples to pears comparison that gives little meaning.

I would prefer to present the results that you have come up with in the form of and odds ratio (OR). The null hypothesis would still be that there is no difference in fire risk between Teslas and the control group. The confidence interval should go both above and below 1, meaning that an OR below one points to a relatively lower risk in Teslas and and OR above 1 a higher risk. Your analyzis would suggest the OR to be something like 0.25 ("4 times less likely") but if you calculate this with a 95% confidence interval (I haven't done the calculations) wouldn't the cofidence interval OR span both above and below 1, for example the OR [95% confidence interval] being 0.1-1.5 due to the small sample size of the Tesla group??? Meaning that what we observe could all just be withing the statistical margin of error? Also, as I said in the above paragraph, whatever we come up with we can discard anyway since our basic data is not really comparable.

Again thinking about medicine, this question is not suited for retrospective statistical quantitative analysis, but much to to and experimental approach: Take 10 Model S and 10 comparable ICE-cars. Drive them on to and over different types of road debris, crash them in identical ways, see how many fires you get. This has been done partly already in the crash testing where there were 0 fires. I'm hoping Euro-NCAP are doing their testing as we speak.

After some years we will have a clearer answer but until then I repeat my claim that the sample size is too small to make any useful conclusions.

This is also why I have said from the very beggining (check my post in the short-term thread that I posted already during Elon's very first interview on CNBC after fire nr 3 where I said oh boy he's coming at this all wrong) that the statistics/numbers route is a bad choice from Tesla since it's a gamble and makes them very succeptible to chance/bad luck in turning the numbers against them.
 
fair enough-
how do you square that with your engineering knowledge of the ModS (in your previous heavy long position, is this something you would not have predicted for example),
assumed Tesla testing results, years of Roadster data (not a design close enough for example?),
results of collision and other testing performed by safety labs, etc.- (they didn't test to real world cases sufficient to pickup on this perhaps?)?

I'm curious why your certainty of this statistical analysis exceeds your certainty of a long stock position, exceeds your belief in Elon and Co to represent same, while concurrently failing to exceed personal safety concerns (via your own ModS experience, fires don't equate to safety perhaps?)?

I think what I hear is that there's enough certainty to warrant no position short or long in stock, but not enough to prevent purchase (or retention) of the car. If everyone followed that course though, you'd have to be long since sales would not be effected

does the certainty of my statistical analysis exceed the certainty of a long or short position? these are two different issues.

a long or short position decision would be based on a number of factors, including valuation and momentum. before, i was sure i needed to be long at a $3 billion market cap with clear positive earnings momentum. i'm not sure i need to be long at $15 billion, regardless of how confident i am (or not) in any other analysis.

now even if my statistical analysis is correct, and indeed there is some kind of problem with the battery in the collisions, the question would be: is that fully discounted in the share price? we've moved down quite a bit off the highs already. and this is not like the bp oil spill where there were tens of billions in potential damages. no one's been hurt. tesla is replacing the cars under warranty. what's the liability here? the greatest liability is the cost of a recall and repair, and the secondary liability would be the short term impact on sales. i'm not convinced either of those are large enough to justify shorting on that basis.

let's say it cost them $10000 to fix 25000 model s. that's what $250 million worst case? unless there was major redesign or retooling involved, how could it cost more than that worst case?
while surely there would be a short term sales impact, would it really matter as tesla is demand constrained anyway? how many billions of market cap has tesla lost since the third fire?

a huge part of why i was doing this analysis is to figure out whether or not there might be a real problem, what the problem might be, and would it be worth buying or shorting tesla. i am confident the data shows there is a problem, or at least a statistical anomaly, around collision related fires. i am confident the data shows no other safety issues related to model s. i am not confident of exactly what the problem is with these batteries and why they are catching fire so frequently recently. i am not confident that there's any meaningful amount of money to be made on the short or long side of this trade. i am confident i'm safe in my model s, unless i get into a collision in which case there may be a fire.

as far as my safety concerns, i let anyone who borrows or drives my car know - if there's an accident, get the *** out of the car and keep your eyes out for potential fire. my statistical analysis shows that's the only rational concern to have.

your question of how come none of this collision fire stuff came up until recently? how was it missed in the tests?

well i don't know enough about the tests to know how it was missed. i can offer a partial statistical explanation for why we may be seeing more now. for that we circle back to my tesla model s "car-years" on the road model.

first let me say that i will have to go back and make a correction, the way i was doing it before i assumed that all the cars delivered in a quarter were on the road the entire quarter. that's not really true. in my revised model i use a linear approximation of the deliveries. for example, previously if i had 5000 cars delivered in a quarter, i counted it as 1,250 "car-years" for that quarter (5000 cars x 0.25 years). that's not correct because the cars are delivered approximately linearly and only an average of 2,500 cars are on the road in the quarter. even that number is too high, because tesla was ramping production (meaning more cars were delivered later in quarters than earlier).

here's the car year model with the correction to nov. 20, 2013:
quarterdeliveriescum vehiclescar-yearscum car-years
9/30/2012250250 31 31
12/31/201224002650 363 394
3/31/201349007550 1,275 1,669
6/30/2013515012700 2,531 4,200
9/30/2013550018200 3,863 8,063
11/20/2013307421274 2,758 10,820

12/31 data estimated






total model s car-years on the road 10,820
the updated car-year model looks like this if projected to end of year 2013:
quarterdeliveriescum vehiclescar-yearscum car-years
9/30/2012250250 31 31
12/31/201224002650 363 394
3/31/201349007550 1,275 1,669
6/30/2013515012700 2,531 4,200
9/30/2013550018200 3,863 8,063
12/31/2013554523745 5,286 13,349

12/31 data estimated






total model s car-years on the road 13,349
take a look at that cumulative car-years column on the far right. see how it's only 4,200 to the end of q2 2013, and then jumps to 8,063 by end of q3? that means the number of car-years on the road increased 75% in the quarter. you see here between june 30 2013 and dec 31 2013 the number of car-years is going to triple almost. they'll have more than twice as much experience in this six month period as they had in the first year since starting production (may 2013).

for readers who are more interested in mileage, this is approximately supported by the tesla shareholder letter data, which shows a roughly 67%+ increase in mileage experience between 8/7/13 and 11/5/13.

8/7/13: "Over 13,000 Model S customer vehicles are now on North American roads and have logged nearly 60 million miles."
11/5/13: "Over 19,000 Model S owners ... have now driven their cars more than 100 million miles."

that's the best reason i can think of as to why we are seeing more problems recently than we did in many months before.
 
I decided to quote your post from way back up at #37 since there seem to be a lot of confused people and this post seems to have been buried. As you can see from this post (for the folks wanting to remove the Mexican fire) even if you only consider the second fire, that put the odds of that event being very remote, and if you do include the 3rd it goes to extremely remote. I think the analysis here is excellent and shows there is greater odds of fire for the Tesla. What is fantastic about the car is that even if there are greater odds, the odds of injury during the event are very low. I agree with Elon and everyone else that the car is very safe. I'm very happy to see Tesla offer to cover fire under their warranty, and that should keep insurance company rates at bay over this issue. I have no problem owning or driving a Model S, nor do I have any hesitation about recommending this car to anyone since it is a technical marvel that is second to none in terms of performance and safety.

understanding the answer to your question requires a firm handle on statistics, and that's why your valid question has a complex answer.

the correct approach to analyzing this type of situation is to use the binomial distribution. here's how it works:

the binomial distribution has only two outcomes, the textbooks often call them success and failure. in my model, a car that has a collision fire in a year of operation is a "success" and a car that doesn't have a collision fire in a year of operation is a "failure".
to get binomial probabilities, you need three pieces of information:
1. p, which is the probability of seeing a successful outcome
2. n, which is the number of observations you are looking at, and
3. x which is the number of "success" outcomes encountered in the n observations.

first let's look at the "p"- the probability of seeing a collision related fire for one year of a typical car's operation. there are extensive statistics on millions of cars, hundreds of thousands of collisions, and tens of thousands of fire in the aggregate nfpa data set. so for me to say that there are 0.0000392 collision related fires per car-year (as i estimated in the kickoff post), i think it's very hard to argue that figure is very far off. there's simply too much data over too many years pointing to that estimate - the estimate is based on almost 130 million cars on the road! from a statistical standpoint what happens is that the estimated error is pretty close to sqrt(p*(1-p)/k) where k is the number of total vehicles for which i am estimating the p. the net result is that the .0000392 should be +/-10% of the actual answer for probability of a collision related fire in a year for an average automobile.

next, consider the "n". i know this is pretty darn accurate as well, as we know how many teslas were delivered and when, and we can pretty easily calculate "car-years" on the road. that's my 13,300. from the standpoint of a binomial distribution, 13,300 is a pretty large "n".

finally consider the "x". that's the 3 fires we observed in a tesla. we can be pretty darn sure that's accurate too (that is there's definitely not less than 3 fires because we have pictures and video of the car burning).

so what i am saying here is that we have a very good handle on the 3 inputs into the binomial distribution, the p, the n, and the x.

under these conditions, i can tell you pretty much *** exactly *** how likely it is that i will see 0 fires, 1 fire, 2 fires, 3 fires, etc.
just put in the following formula into an excel spreadsheet:
=binomdist(<<insert number of observed fires>>,13300,0.0000392,FALSE)

if you do this, you'll get these results (for various numbers of observed fires):
observed fires probability
0 0.5937
1 0.3095
2 0.0807
3 0.0140
4 0.0018
5 0.0002
now here's how you can interpret the data - you can ask, what is the probability i would see 0 fires in the teslas up until now?
the answer is 59.37%
how about exactly one fire by now?
the answer is 30.95%
how about one or fewer fires by now?
the answer is 0.5937+0.3095 = 90.33%

and the important question: what is the probability i would see less than 3 fires?
that's the sum of the 0+1+2 fire values: 0.5937+0.3095+0.0807 = 0.9839%

that is, if tesla model s were as likely to have a collision fire as an ice automobile, there's a 98.39% chance that we would have seen 2 or less fires by now. this is virtually a mathematical fact.

at this point, i am realizing i have an error in my original post which i will soon fix - i came up with a probability that was a bit too high as i used 3 or fewer fires (i should have used 2 or fewer). i will go back and fix it.

that's why this third fire was so important, when there were 2 fires you still had an 8% chance of seeing that many fires. but now with 3, the probability is dropping of sharply that this is just a random variation we're seeing (of course that's still possible, it's just that the odds are now under 2%).

i'm really not sure how else i can express this view more clearly. if it's still confusing perhaps someone else who knows can chime in.

the sample size is not 3 - it is 13,300. the number of observed outcomes is 3. assuming my model inputs are correct or very close, the binomial distibution should correctly account for all of the facts properly in calculating probabilities.

- - - Updated - - -



i haven't seen such a detailed level of data in my research, so i did the best i could with what i had =)

however, you could say that no matter what the separation is, it would almost certainly be worse for tesla. that's because collisions presumably include multi-car crashes + hitting road debris. so the number of fires related to hitting road debris is surely less than the number i gave, and that would make model s look even worse. the probability of a road-debris collision fire would be even lower, and then the odds of us seeing 3 model s fires being a random fluke would be even closer to zero.
 
OK, with all of one college statistics class under my belt, I fear to venture into this head, but after the 3rd fire, I ran across the following report published by the AAA Foundation on road debris related accidents. Its 1990 data, but it would still seem to be somewhat useful. The TL;DR summary is the attribute 25,000 accidents each year to road debris with approximately 80-90 resulting fatalities. Not sure who's viewpoint this supports, but here you go: https://www.aaafoundation.org/sites/default/files/VRRD.pdf

O

thank you for the link. this table 7 was interesting - it shows the estimated number of all crashes that involve debris is less than 1%.

Table 7. Estimated VRRD Tow-Away Crashes in the United States, based on CDS Database,
1997–2001 (Weighted Data)
Year
Total
Number of
Crashes
(millions)
Estimated
Number of
VRRD
Crashes
Standard Error Percentage of Crashes
that involve VRRD
1997 2.8 24,985 13,062 0.89
1998 2.6 5,438 2,897 0.21
1999 2.6 2,873 1,302 0.11
2000 2.5 14,128 6,033 0.57
2001 2.5 12,096 8,049 0.48
All 13.0 59,520 17,196 0.46
 
does the certainty of my statistical analysis exceed the certainty of a long or short position? these are two different issues.

a long or short position decision would be based on a number of factors, including valuation and momentum. before, i was sure i needed to be long at a $3 billion market cap with clear positive earnings momentum. i'm not sure i need to be long at $15 billion, regardless of how confident i am (or not) in any other analysis.

.
.
.

that's the best reason i can think of as to why we are seeing more problems recently than we did in many months before.

excellent- thanks for that detail; clears up a misconception regarding the effect on the stock and your position.
I think the increased car-years is a good possible answer. I'm still at a loss to reconcile to the testing which should have simulated these kids of car-years.

How would the new car-years basis (linear intra-quarter treatment) form to predict the time we would expect to see the next collision-fire? Seems like with the high certainty of the statistical model, that would be an easy and reliable prediction to make. Might be a good test for the model, if nothing else to register possible effects of the recent height clearance
 
I think that luvb2b's calculations are fundamentally wrong, because he ignores the fact that 2 fires happened after running over a metal object that can pierce the car's underbelly. One fire happened outside of US and should be excluded from the calculations, because luvb2b compares these fires to the US data.

MS had a clearance much lower than an average car, lower than most sports cars. So if there's a trailer hitch on the road, it's possible that 100 cars will drive over it before MS finally catches it with it's battery.

MS was much more likely to catch a piece of road debris than an average car.

If you compare MS fires to all other car fires resulting from hitting road debris, MS was probably more likely to catch fire than an average car.

But because there were no other types of MS fires in US, it's save to say that in any other circumstances (collisions or whatever) MS is much less likely to catch fire.

Now when the last firmware increases the clearance, MS has the same chances of catching road debris as most other cars. So now it will have much less collisions with debris.

What does this mean? From now on MS will have much less chances to catch fire than an average car.

you know what i love about the investing business? one's speculations are proven true or false over time.

you may be right, but isn't it only the air suspension which has adjustable ride height? what about all the non-air suspensions?
 
Great post. It covers most of what I have said, though in greatly more detail. I think the really significant point in all of this is fires/undercarriage strike. There is not way to get stats for that for ICE vehicles, but based on what the stats for fire/accident show, there is probably an even more significant trend there. The point of your analysis and my own is not to draw ire from Tesla fans or denigrate the car, it is to demonstrate that in a very particular circumstance, the Model S may be prone to damage that leads to fire. This is nothing against the car, EVs, or Elon Musk. It is simply a statement of fact based on the best available evidence. If no changes are made the MS and these incidents stop occurring, then we'll know more. If the opposite happens, it just props up current suspicions. This is precisely why the NHTSA is investigating. It wants to know if this is a problem or not. We'll know soon enough.
 
As SwedishAdvocate so notably brought back, and I will bring back again to the top - Mario Kadastik's real assessment...

"Please everyone stop [drawing erroneous conclusions from (My edit.)] the statistics. The ICE fires are in the thousands and their distribution is governed by the normal distribution that most of your statistics are based on. The Model S statistics are so low that it's governed by Poisson statistics and that has completely different characteristics. I deal with low probability events daily (Higgs search at LHC) and have had to handle the differences and you can't believe how much difference there is. Your math here has error bars that are so huge that you cannot draw any conclusions really. In Poisson statistics 0-2 events are statistically inseparable so even if you expect 0 events and observe 2 you cannot claim disparity between the two measurements. With three you start to get somewhere, but only if you really expected 0 in the first place. If you expect even one (or worse ca 3), then one to about six events are fully compatible (or one to ten). You can start using your normal statistics when the number of incidents expected is largish i.e. my statistics teacher used to say that 30 and infinity are about the same, it's not quite that simple, but around that region the Poisson starts to converge towards the normal distribution..."

Ugh. Another Model S fire - 2013-11-06 - Page 40

What do you do for work? Just passing time waiting for updates from Tesla :) - Page 36

i am still waiting for mr kadastik to weigh in.

estimates of means for the ice fires are governed by the normal distribution, he is correct.

the low incidence model s fires can be addressed by a binomial distribution. your friend mr kadastik works in continuous time space, and that's why perhaps he is missing the application of the binomial distribution in estimating probabilities here.

but, as i said, i would love for him to weigh in.
 
Fire 2 was not intentional. It was the result of an accident. To be intentional, the person has to willingly bring a flame to the car or purposefully damage it with the intent to start a fire.

- - - Updated - - -

Why is that?
 
Your conclusions are invalid because you used flawed data.

all data is flawed in some ways. if you're waiting for a perfect set of data, well it'll be like waiting for the perfect man (or woman)... you'll wait a long time.

can you show me a better set of data and better analysis?

3 tesla batteries have caught fire shortly after collisions. that is a fact.

the best data set available indicates that this is anomalous number given how many teslas have been on the road and for how long.
 
Fire 2 was not intentional. It was the result of an accident. To be intentional, the person has to willingly bring a flame to the car or purposefully damage it with the intent to start a fire.

- - - Updated - - -

Why is that?

Gee, I don't know. Driving drunk seems like an awfully intentional action as does speeding.



yeah- it's really 2 discussions. It doesn't fit the Intentional category as defined in the data used by Luvb2b - I think I raised this point to start with; My intent was not to place it in that category, because I knew it didn't actually fit the data. My intent in describing it as Intentional is similar to Krugerrand- It might as well be Intentional for the purposes of only 3 Observations of fire in the model. Because using it still implies that 33% of observed ModS collision fires will require a drunk driver deciding to drive through a concrete wall. It's the intent of the driver to conduct himself beyond the statistical profile of the model (when 33% of the observation depends on it).

I think Luvb2b has shown though, even if you take it out, there's a problem to consider educating about or fixing- so it may be a mute issue for the next events
 
I think Luvb2b has shown though, even if you take it out, there's a problem to consider educating about or fixing- so it may be a mute issue for the next events

yep, back to post #37.

The calculation is not the odds of Tesla having a fire, its the odds of any population of cars with 13,300 car-years. You could go out on January 1st and put a red X on 13,300 cars at random and the odds of one of those cars catching fire in the next year (due to a highway collision) are:

0: 59.37%
1 or less: 90.33%
2 or less: 98.39%

so the first fire was a 60/40 split, so pretty random, the second one put it to 90/10 which becomes significant, and if you include the third you get to 98/2 which is very unlikely. (it means that there is something odd about your data-set if you have somehow got three out of your 13,300 sample)
 
Last edited:
you know what i love about the investing business? one's speculations are proven true or false over time.

you may be right, but isn't it only the air suspension which has adjustable ride height? what about all the non-air suspensions?
All the vehicles so far affected have air suspension and were also P85 models. The non-air suspension rides higher because there is no auto lowering.

This again makes it clear to me there really is two probabilities mixed in. The probability of hitting debris (which would be helped by raising suspension) and probability of catching on fire after hitting debris (which would need more armor or a pack redesign).

- - - Updated - - -

Remember that the NHTSA is only considering the U.S. fires and still feels the need to investigate.
The NHTSA has stated they would be more interested in EV/plug-in fires than other types of fires (which is why they also investigated the Volt and Karma fires). And NHTSA did not say they aren't figuring in the Mexico fire as reason to investigate, but rather than it's outside their jurisdiction (so they can't investigate it even if they wanted to).
 
this whole discussion is a waste of neurons, the car is still the safest car on the road. These fires are non-events as they are not spontaneous explosions like the ones that happen in ICEs.

Who the heck cares if they happen or not as long as no one gets hurt and Tesla pays for the 1 in X thousand cars it happens to?