Tesla ranks 2nd worst in Consumer reports reliability survey

[john5520]

I think that we must accept that "reliability" no longer means "Fix Or Repair Daily," or "Found On Road Dead," things that for the most part don't occur much anymore, Instead, the word has evolved into something else. Something that when said, means what the person saying it means, and it is up to us to figure that out.

I think most consumers still regard reliability in the traditional sense. However, it's easier to just take things at face value from a publication they trust rather than do more research.

 

[john5520]

When it was all over the news and many were asking if it was possible they tried it and answered the question. They also suggested a simple way to fix it (inside camera). That was not excessive. That was responsible research and reporting.

Demonstrating how to defeat a system is not responsible especially knowing there's a segment of the population that would try it. I agree with the addition of the internal camera, though, I think their test was fueled more by an agenda rather than in good faith.

 

[daniel]

It would be useful to have two statistics for each make, model, and year:

1. What percentage of cars are expected to need a non-critical repair each year, defined as a repair that does not prevent the car from operating safely and can be scheduled at the owner's convenience; and,

2. What percentage of cars are expected to need a critical repair each year, defined as a problem that prevents the car from being operated safely and must be performed before the car is driven again.

In the case of (2) it would be useful also to know the average length of time the car is out of service. (Time to get the needed parts and make the repair.)

 

[SO16]

It would be useful to have two statistics for each make, model, and year:

1. What percentage of cars are expected to need a non-critical repair each year, defined as a repair that does not prevent the car from operating safely and can be scheduled at the owner's convenience; and,

2. What percentage of cars are expected to need a critical repair each year, defined as a problem that prevents the car from being operated safely and must be performed before the car is driven again.

In the case of (2) it would be useful also to know the average length of time the car is out of service. (Time to get the needed parts and make the repair.)

Now THIS would actually be useful to customers.

 

[Bouba]

I’d go further....what was your experience like? Did the staff make you feel like you were disturbing them? Did you get a loaner car? Did it require you taking a lot of time off work etc etc.
At the end of the day a major fault that is repaired promptly, efficiently and politely is better than getting the run around over a minor problem

 

[Blue Stuff]

Report on it, sure. But can accomplish the same thing without a “how to” video. All they would need to say is that the driver monitoring is too easily defeated for our recommendation.

Is it “responsible reporting” if the media does a step by step on how to make a dangerous explosive with simple and easily accessible items? No.

They aren't the media. They are a testing company so they tested and reported their results. If they didn't say how they tested everyone would call them out. They do the same thing for things like deep fryers.

 

[David29]

And here we are. The raw data itself is fine, the questionable part is how that data (or lack thereof) is interpreted to come up with their final reliability score.

Here is a link to a detailed article about how CR collects and analyzes their data on reliability. (I cannot tell easily if the article is available for non-members, because I am a member so I can see it, but it may not work for everyone, sorry.) Consumer Reports' Car Reliability FAQ
CR makes it clear that reliability is based on reported "problems," with problems ranging in severity, not only on problems that cause a car to be out of service. I think most people would consider a car to be less reliable if it needs to go to be repaired frequently, not just that it won't run. Over the many years in which I drove Mercedes-Benz cars, which used to have a reputation for very high reliability, my cars left me by the side of the road and unable to drive only once or twice (leaving aside accidents and tire failures). But they sure spent plenty of time in the shop for fixing things and cost me a lot of money. My family's Hondas and Toyotas were much more reliable in the sense of needing fewer repairs and costing less money to repair, but they did occasionally fail to operate.

Here is a quotation from the CR article, explaining how CR says they assess the relative importance of various types of problems:
"Are All Problems Considered Equally Serious?
Engine major, engine cooling, transmission major, and drive system problems are more likely to take a car out of service and to be more expensive to repair than the other problem areas. Consequently, we weight these areas more heavily in our calculations of model year overall reliability verdict. Problems such as broken trim and in-car electronics have a much smaller weight. Problems in any area can be an expense and a bother, though, so we report them all in the reliability history charts."

One other point that might cause confusion -- CR presents reliability ratings of existing cars, with color-coded charts showing the data for each car make/model combination in 17 categories. So that is fairly straightforward. But they also do predicted reliability on new models for which there is little or no data yet. I suspect that news stories about these ratings sometimes fail to distinguish between the two.
(And on another point of reliability versus consumer preference -- We also had a Jeep Grand Wagoneer along the way, and it probably ranked low in reliability -- the carburetor and ignition system were trouble spots as I recall -- but we loved it! It was a fantastic trailering vehicle.)

 

[sleepydoc]

They aren't the media.

Actually, they are. CR does testing, data collecting and reporting. They also report on consumer issues apart from their testing data and that makes them a media company.

I agree with @john5520, reporting that the system can be defeated is one thing, but describing in detail how to defeat it is another thing entirely. I also found that article alarmist, excessive and biased in its conclusions. The point isn’t whether a safety feature can be defeated. The features are there to keep honest people honest. If you have an irresponsible driver, they will be irresponsible regardless of what you do. Tesla has more than enough warnings and safety systems in place. Blaming Tesla is completely ignoring the driver’s (primary) role in all of this.

Edit - if they were consistent, they would be saying it’s the car companies’ fault for not putting limiters on cars rather than the drivers’ fault for speeding.

 

[sleepydoc]

My question is, What was the part that was deemed "not reliable" by these drivers?? Does it not start? Won't turn? Headlights not work? Just calling it "unreliable" doesn't do it for me. I have owned four Teslas and I have found them JUST AS RELIABLE as my Toyotas, all of which never had any problems. Toyota had some recalls, so did Tesla. But that didn't make them "unreliable". Unreliable means you can't depend on them to get you down the road, and that has never ever happened to me, with Toyotas OR Teslas. And there's nothing tribal about that.

As has been stated multiple times in this thread, CR provides a detailed breakdown of the different areas that they rate, what specific parts and types of repairs are included and how those individual areas rate. Like others have said, anecdotes are not reliability data and saying the report is worthless because you didn’t bother to read and understand it says more about you than it does the report.

I think that we must accept that "reliability" no longer means "Fix Or Repair Daily," or "Found On Road Dead," things that for the most part don't occur much anymore, Instead, the word has evolved into something else. Something that when said, means what the person saying it means, and it is up to us to figure that out.

I think it’s more of the “severity” of issues vs number of issues. It seems the media like to make it “sound” like Tesla has all these major issues when in reality they are minor. The term “reliable” I think is misleading. It should simply be number of issues. If they were major issues I think Tesla customer satisfaction would be much lower.

@DrGriz is right - In years past it was commonplace to get stranded. Fortunately for all of us, the overall reliability of cars has improved dramatically over the past decades. Even the worst Chevy today is probably more reliable than a Honda or Toyota from 20-30 years ago. Beyond that, one needs to remember that these are statistics. They essentially give you the odds of having a problem, not a guarantee. In the end the rating ends up showing how a given car compares to other cars.

@SO16 - CR does weight severe problems more heavily but Teslas have some major issues but a disproportionate number of minor issues, pushing their ranking down. The problem is, no one seems to be able to read past a headline these days. If you read the full report, you can get a lot of the nuances. If all you do is look at the big number, you miss that. The relationship between satisfaction and reliability has also been discussed at length in this thread.

 

[DrGriz]

For those who are not members, here is an excerpt from CR's page describing their reliability evaluation process. I think it is worth noting that they only have information for one model year for the Model Y. Also, they adjust for the age of the owner, but I could not find out what their precise methodology for doing that is. I guess they are saying that Millennials are pickier than Boomers. Do they double the percentage of problems for Boomers, or do they cut them in half for Millennials? Inquiring minds would want to know.


The reliability data comes from our Auto Reliability Surveys of Consumer Reports members. In all, we received responses on over 300,000 vehicles in our latest surveys, detailing 2000 to 2020 models.

Consumer Reports members reported on problems they had with their vehicles during the past 12 months that they considered serious because of cost, failure, safety, or downtime in any of the trouble spots included in the table below.

The scores in the charts are based on the percentage of respondents who reported problems in each of the 17 trouble spots. Because high-mileage cars tend to encounter more problems than low-mileage cars, problem rates were standardized to minimize differences due to mileage. As part of the continuous improvement process, new this year CR is adjusting for the vehicle owner’s age, based on our findings that older owners are more likely to report fewer problems.

 

[Bouba]

For those who are not members, here is an excerpt from CR's page describing their reliability evaluation process. I think it is worth noting that they only have information for one model year for the Model Y. Also, they adjust for the age of the owner, but I could not find out what their precise methodology for doing that is. I guess they are saying that Millennials are pickier than Boomers. Do they double the percentage of problems for Boomers, or do they cut them in half for Millennials? Inquiring minds would want to know.


The reliability data comes from our Auto Reliability Surveys of Consumer Reports members. In all, we received responses on over 300,000 vehicles in our latest surveys, detailing 2000 to 2020 models.

Consumer Reports members reported on problems they had with their vehicles during the past 12 months that they considered serious because of cost, failure, safety, or downtime in any of the trouble spots included in the table below.

The scores in the charts are based on the percentage of respondents who reported problems in each of the 17 trouble spots. Because high-mileage cars tend to encounter more problems than low-mileage cars, problem rates were standardized to minimize differences due to mileage. As part of the continuous improvement process, new this year CR is adjusting for the vehicle owner’s age, based on our findings that older owners are more likely to report fewer problems.

Interesting, so this is their first year that they have adjusted for age. And apparently Tesla owners are older. So it could be completely wrong. They may have grossly overestimated how being old affects the reporting.
For example, Tesla owners are possibly (probably) more technologically savvy than the general public. Also the age range might be skewed. For example the Tesla owners maybe, on average, older but may contain less extremely old (who might be more conservative in their car buying and only buy brands they are familiar with).
I maintain that this report is not infallible

 

[DrGriz]

Interesting, so this is their first year that they have adjusted for age. And apparently Tesla owners are older. So it could be completely wrong. They may have grossly overestimated how being old affects the reporting.
For example, Tesla owners are possibly (probably) more technologically savvy than the general public. Also the age range might be skewed. For example the Tesla owners maybe, on average, older but may contain less extremely old (who might be more conservative in their car buying and only buy brands they are familiar with).
I maintain that this report is not infallible

I would agree, especially that the assumption around generational differences applies to Tesla owners, since they have already self selected as a different group. For instance, half of my patients, who range from 50 to 100 years in age, can barely use a smart phone and wouldn't, for that reason ever purchase a Tesla. So those in that group who own Teslas are already quite different than whatever norm CR developed for ICE car owners.

I'd also like to know how they "correct" for high milage vs low milage in EVs. It appeared when I looked at them that most of the issues where the Teslas did poorly in those 17 categories were initial quality problems, their "high-mileage cars tend to encounter more problems than low-mileage cars" assumption would be likely actually be reversed for Tesla. We have no way of telling how much they applied this assumption and whether or not it skewed the ratings.

 

[DrGriz]

On the other hand, they included in the Model Y ratings categories such as exhaust problems, transmission issues, and engine cooling / radiator leaks, in which, of course, the Tesla excelled. Of course, rating those things is nonsense since there are none in Teslas.

It's an apples vs horses reliability rating. I'm not sure that they should be including ICE vehicles and EVs in the same survey. Or maybe they should throw refrigerators into the same review.

 

[Bouba]

On the other hand, they included in the Model Y ratings categories such as exhaust problems, transmission issues, and engine cooling / radiator leaks, in which, of course, the Tesla excelled. Of course, rating those things is nonsense since there are none in Teslas.

It's an apples vs horses reliability rating. I'm not sure that they should be including ICE vehicles and EVs in the same survey. Or maybe they should throw refrigerators into the same review.

Can I add, that when I first saw your avatar, I didn’t realize it was a house or barn. In fact I thought it was a Cybertruck!!
So add to your observations, that half the people between 50 and a 100 have poor eyesight

 

[DrGriz]

Can I add, that when I first saw your avatar, I didn’t realize it was a house or barn. In fact I thought it was a Cybertruck!!
So add to your observations, that half the people between 50 and a 100 have poor eyesight

Pretty sure Elan saw my barn and was inspired by it!

 

[Null860]

The vast majority of owners of all car brands have had no significant problems with their cars. A few people saying "I've had no problems with mine" is meaningless when discussing reliability overall.

In addition, cars are much more reliable than they used to be. Pointing out that the car somebody had in 1970 was way more unreliable than their Tesla also tells you nothing about the relative reliability of today's cars.

And how can anyone claim an anti-Tesla bias when the most popular Tesla gets a good rating?



Other than one Yahoo article posted upthread, is there any evidence that this is what CU did? Or is this just something that's been claimed in this thread?



I guess they changed their name. Consumers Union was the publisher of Consumer Reports until (unbeknownst to me) Consumers Union changed its name to Consumer Reports.

My apologies for using the old name. I'm an old fart and I still think of them by their old name.

Read CA's report...

It's been a problem w/CA's ratings since at least the 70's...

 

[daniel]

Read CA's report...

Do you mean CR? I did read their report. (I've been a member for longer than I can remember. Back before the Internet.)

 

[sleepydoc]

On the other hand, they included in the Model Y ratings categories such as exhaust problems, transmission issues, and engine cooling / radiator leaks, in which, of course, the Tesla excelled. Of course, rating those things is nonsense since there are none in Teslas.

It's an apples vs horses reliability rating. I'm not sure that they should be including ICE vehicles and EVs in the same survey. Or maybe they should throw refrigerators into the same review.

Like I said, EVs are a new paradigm. Including items such as 'exhaust system' for EVs seems ridiculous at first blush, but then, how do you take it out? If you have fewer categories for EVs than ICE vehicles then the problem rate would go up. If you separate out all EVs you're only comparing them to like vehicles, but then that precludes any comparison with any other vehicle. One of the benefits of an EV is there isn't an exhaust system to fail, so I think keeping it in the ratings is the proper choice. Yes, all EVs will do well in this category, but that's part of their advantage.

Of course, it also begs the question, how do you classify battery issues? ICE cars don't have a Li battery to fail so they'd do better here. I would include the battery as part of the 'drive train' in an EV, but I don't know what, if any changes CR made in this area.

As far as adjusting for age goes, that is part of statistical analysis. Any good study will try to identify and account for confounding variables. That is what CR is doing here. Part of their data set is age, so they can tell if someone is 55 or 85. Is it possible that a 60 year old Tesla owner is more likely to report problems than your average 60 year old, making the correction unnecessary? Yes. Of course, given the average Tesla owner's loyalty, it's also possible that they will be more likely to underreport problems.

Tesla Owner Demographics: Age, Gender, Ethnicity, Income, Children — EV UNITE

The demographics of an average Tesla Owner in the United States. Including Age, Ethnicity, Gender, Income, Children, Home Ownership.

www.evunite.com

It appeared when I looked at them that most of the issues where the Teslas did poorly in those 17 categories were initial quality problems

Yes - pretty much every problem in a one year old car would be an initial quality problem, but CR is comparing 1 year old cars to 1 year old cars so it doesn't matter. What their data shows is Teslas have a higher rate of these issues than other cars.

 

[john5520]

Here is a link to a detailed article about how CR collects and analyzes their data on reliability. (I cannot tell easily if the article is available for non-members, because I am a member so I can see it, but it may not work for everyone, sorry.) Consumer Reports' Car Reliability FAQ
CR makes it clear that reliability is based on reported "problems," with problems ranging in severity, not only on problems that cause a car to be out of service. I think most people would consider a car to be less reliable if it needs to go to be repaired frequently, not just that it won't run. Over the many years in which I drove Mercedes-Benz cars, which used to have a reputation for very high reliability, my cars left me by the side of the road and unable to drive only once or twice (leaving aside accidents and tire failures). But they sure spent plenty of time in the shop for fixing things and cost me a lot of money. My family's Hondas and Toyotas were much more reliable in the sense of needing fewer repairs and costing less money to repair, but they did occasionally fail to operate.

Here is a quotation from the CR article, explaining how CR says they assess the relative importance of various types of problems:
"Are All Problems Considered Equally Serious?
Engine major, engine cooling, transmission major, and drive system problems are more likely to take a car out of service and to be more expensive to repair than the other problem areas. Consequently, we weight these areas more heavily in our calculations of model year overall reliability verdict. Problems such as broken trim and in-car electronics have a much smaller weight. Problems in any area can be an expense and a bother, though, so we report them all in the reliability history charts."

One other point that might cause confusion -- CR presents reliability ratings of existing cars, with color-coded charts showing the data for each car make/model combination in 17 categories. So that is fairly straightforward. But they also do predicted reliability on new models for which there is little or no data yet. I suspect that news stories about these ratings sometimes fail to distinguish between the two.
(And on another point of reliability versus consumer preference -- We also had a Jeep Grand Wagoneer along the way, and it probably ranked low in reliability -- the carburetor and ignition system were trouble spots as I recall -- but we loved it! It was a fantastic trailering vehicle.)

Thanks for posting this. In the article they said they believed their data tracked well with other reliability and repair studies, however, I'll have to disagree with that after comparing it to JD Power results over several years. They disagree on more brands than agree from what I've seen. This is one of the problems I have with the results, and it shows that surveys such as these aren't as valid as they would like you to believe.

I do have one question regarding the results per vehicle. I understand each category has a list of parts that could apply. However, do they say per vehicle which part in the list of possibilities was faulty? I was very tempted today to purchase a month's subscription for $10 to see for myself.

 

[john5520]

I'd also like to know how they "correct" for high milage vs low milage in EVs. It appeared when I looked at them that most of the issues where the Teslas did poorly in those 17 categories were initial quality problems, their "high-mileage cars tend to encounter more problems than low-mileage cars" assumption would be likely actually be reversed for Tesla. We have no way of telling how much they applied this assumption and whether or not it skewed the ratings.

I agree. But here's the bigger issue I see in a nutshell: Joe User looks at the Consumer Reports freebee results shown on the web, see's Tesla's reliability rating close to the bottom and think the car is unreliable in the traditional sense. He's less likely to investigate further or subscribe for more "details". If he's adamant about getting one though, he would hopefully turn to Google for more info and find this or another forum where more useful information resides.

How many would actually investigate further? I'm guessing a much lower percentage than those who would take that result at face value.