The problem is if that the right comparison to make (as you can pick from many different types). From a safety standpoint, the driver cares about if the car will catch on fire after they hit debris and that null hypothesis does not tell you the answer (as it mixes in probability of collision in the first place which is affected by other factors, some of which are driver choices, like speed and following distance).
The null hypothesis that does is this: the risk of fire in Tesla Model S collisions is no greater than the risk of fire in the average ICE car collision.
And we don't have the data to support this without knowing the collision rate of Model S (currently the assumption is it's the same as ICE vehicles, which may not be a good assumption).
for reference my original null hypothesis:
the risk of a collision related fire in a tesla model s is no greater than the risk of a collision in the average ice automobile.
the null hypothesis you propose:
the risk of fire in Tesla Model S collisions is no greater than the risk of fire in the average ICE car collision.
first off, i want to compliment you on one of the most thoughtful responses i've received on this thread. you are correct that the null hypothesis you are presenting is a better choice. the difference is so subtle i'm not sure many of the readers will see it, so i'll rephrase your null as the question: "once a model s or an ice get into a collision, which is more likely to catch fire?"
and your thought here is correct, we don't have access to data on model s collisions to directly answer the question.
i had thought about a few of these issues as i did my research.
i think the first thing i realized is that if i could know that the odds of a model s collision were less than or equal to the odds of an average ice collision, then my test would err in favor of tesla.
what i found is that there is a very powerful effect in auto insurance underwriting from education and occupation. that is either highly educated or blue-collar employed individuals have 15-20% fewer claims than the average. there was a whole controversy about this because geico was using these factors to adjust insurance premiums, and there was a big stink about it being a proxy for race. for the data table, refer to page 22 of this study (page 23 of the pdf):
http://www.state.nj.us/dobi/division_insurance/pdfs/ed_occ_april2008.pdf
"This information demonstrates that Occupation Groups 1 and 2 have better loss experience than the others, and that drivers with a Bachelor’s or Master’s degree
are similarly less risky than the population generally. The differences are
statistically significant and thus sufficient under current insurance statutes to be
reflected in the rates charged to these driver groupings.
Based on this data, for example, individuals in Occupation Group 1 generate
about 15% less claims than average drivers, while individuals in Occupation
Group 5 generate greater than 25% more claims than average drivers. Similar
results are documented in the loss ratios for groups with various levels of
education."
i think it's reasonable to assume that the model s drivers are likely to be either more educated or have a white collar job vs. the average ice driver. for example, it's probably safe to say a lot fewer teenagers are driving teslas than ices.
unfortunately that's all i could find that would relate to the likelihood of a model s collision vs a regular ice collision.
however, another thing i learned along the way is that the nfpa statistics cannot distinguish between a pre-collision fire and post-collision fire in most cases. that is, did the car catch fire because of an electrical malfunction, then the driver panicked and had a collision? or did the driver have a collision, and then the car caught fire? because mechanical and electrical failures are the primary causes of ice fires, there's likely a significant pre-collision fire effect in the ice collsion-fire data. this was discussed in some of the source documents i read.
the national data is such that it simply cannot distinguish the scenario of whether the caught fire before or after the crash. so, it's not possible to directly address the question you posed.
a further consequence is that the nfpa collision-fire data is for sure higher than the actual frequency of post-collision fires. in the 3 model s fires, they were all post-collision fires. so the way i compared these statistics, they are skewed in favor of tesla because what we're interested in, and what i should be using is the rate of post-collision ice fires.
it gets back to what another poster said, we do the best with what we have.
- - - Updated - - -
>LUVB2B
Firstly, its erroneous statistics to include Mexican EVs but exclude Mexican ICEs statistics in the analysis. Either adjust the ICE stats to include Mexican ICEs or adjust the EV stats to exclude Mexican EVs. Mexico is a Nissan country and shares many vehicles with USA, its wrong statistics to mix developed and developing world accident safety stats, even if the vehicles are identical.
you've raised a valid concern here. i had thought about this too from a few different angles, and i'll share with you why i included the mexico crash. i can also share with you what happens if it is excluded.
first, reliable data is not available for mexico, so let's just forget about trying to adjust the analysis for mexico.
so the only question is, do we include the mexico crash in the count, or do we leave it out.
i decide to include it for these reasons:
(a) the car must have been built to usa safety specifications. it must have been an "american" tesla for lack of a better word.
(b) the driver did not collide with another vehicle, and that would certainly complicate the situation.
(c) the driver hit inanimate objects like barriers and trees, which would be relatively similar in different countries.
(d) i think the question of interest is "are all tesla model s less likely to have fires in a collision than the average ice in the united states?" if that's the question, i believe it's ok to use all tesla model s regardless of geography.
however, if you wanted to throw it out, you could easily adjust my analysis and do so. at that point the null hypothesis is still rejected but only at a 10% significance level, as there is just a little over a 90% chance that you shouldn't even see 2 crashes (using my collision-fire probability).
>General
Fleet on road (all ages) is very relevant for regulatory bodies, particularly in regard to recalls. Elon is very correct to use on-road fleet stats in that context. Ie IF Tesla Model S has a lower fire rate than half the vehicles on the roads, then that half of vehicles on the roads should be recalled before Tesla would be recalled. If its 95% etc...
i don't think a blanket recall is going to be issued unless the investigation reveals a meaningful problem with the undercarriage or battery placement. all of the data supports elon's claims that overall the model s is safer than ices. it's just this nuisance of the post-collision battery fires where they seem to be having a problem.
VMT, vehicle miles traveled seems to be how NHTSA would analysis this, so safety per 100M VMT is a key metric.
http://www-nrd.nhtsa.dot.gov/Pubs/811701.pdf http://www-nrd.nhtsa.dot.gov/Pubs/811845.pdf As Tesla model S passed the 100M VMT a level of analaysis can be observed, the Tesla is very safe for its occupants. No fatalities have been reported, and while 'injury' has different definitions, No injuries have been reported where par value would expect around 80 injuries.
well as i stated in my original post, all the data supports elon's overall safety conclusions (although he used exaggerated claims, his conclusions were correct). so there's no need for us to spend time on model s overall safety questions, we are in agreement there.
i am looking at just collision fires.
the problem with using the millions of miles traveled is that the nfpa data on collision-fires is all based on time, not mileage. so to meaningfully address collision fires, the only way i see is to use time on the road ("car-years"). if i were to add in the mileage analysis based on the federal highway administration's estimates of miles traveled, it would introduce more error into the analysis. not every car maker has the luxury of tesla, where tesla can know at every instant how many miles each of their cars on the road has traveled.