Do either CR or JD power release how they conduct their surveys? Sample sizes, questions asked, statistics about the data responses?
Is reliability a car breaking down or needing some minor service? Is there a difference if it requires you to take it to the dealer right away or you can operate the car just fine until you have time?
What about if you want to keep the car for 10 years?
I’ve never seen any of this from either.
The best I found from J.D. Power was on this page:
Ratings Methodology | How J.D. Power Ratings Work | J.D. Power. They state that they survey ‘verified’ owners, but don’t give any information on how they select them, how many people they survey, etc. It’s also important to note that J.D. Power has 2 different surveys, an Initial Reliability survey and a longer term, 3 year one. I haven’t seen anything describing J.D. Power’s funding; from what I’ve read, they sell their reports to companies but I don’t know if they receive any other funding. Selling reports is fine, but receiving direct funding from the companies you are evaluating raises questions about bias and impartiality.
Consumer reports is very clear that they do not accept funding from outside sources; their funding comes from members. They base their auto data on responses from member surveys. More info on their methodology can be found on these two pages:
Learn more about Consumer Reports' ratings methodologies and test protocol updates.
data.consumerreports.org
Consumer Reports answers common car reliability survey questions to explain the thought process and science behind the ratings.
www.consumerreports.org
It’s important to note that ’reliability‘ means different things at different points in a car’s lifespan. There’s a well known model described as the
bathtub curve describing failures. Also, there are different types of problems. Minor mechanical defects such as misaligned panels, etc will show up early but not late. Beyond a few years out, failure is more likely due to component design and quality. Early surveys and even 3 year surveys will not give much useful information on this. This also means that early surveys may have little to no relevance on how a car will age over time. Car Makers like to tout the J.D. Power Initial Quality survey. A high rating looks good but this is more an indicator of manufacturing quality control and and the quality checks that occur prior to delivery than of actual build quality. There have been plenty of cars that scored highly on this survey that were notorious for long term problems.
The other thing to note is that there are (or at least should be) very few problems in the first years of owning a car. This means that it takes relatively few problems to sink a car to the bottom of the list. Later on, a reliable 5 year old car will still have more problems than a new car. Consumer Reports tries to take this into account in their ratings. How they do so has changed in the past and I haven’t looked recently to see exactly how they are doing it now.
Tesla’s manufacturing issues have been well known and reported so it’s really no surprise that they performed poorly on the initial quality survey. They’ve also had to deal with extremely rapid growth (and do it in a pandemic with supply shortages,) something that would be difficult for any company. We’ll see how their long term reliability stacks up - I’m hopeful that their design is sound, even if they have issues with initial quality control. (I bet $55k on it!)
For long term reliability, Consumer Reports is probably about the best. One needs to realize that this is intrinsically trying to predict the future of a moving target. For cars like the Model Y, all you can do is go off of Tesla’s past history with other cars which may or may not apply to the Y. Even for an established model, changes to the design and manufacturing process mean that the results for past years may not apply to the current model.
