Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Tesla.com - "Transitioning to Tesla Vision"

gearchruncher

Active Member
Sep 20, 2016
2,510
3,316
Seattle, WA
Apparently, there are test procedures. This is what I could find in my cursory search:
"August 2014 draft AEB procedures" - That is hardly a good reference that in 2021 there are solid, industrywide tests that NHTSA uses to evaluate AEB. They also only used those on 4 vehicles and modified the tests during that procedure. All the vehicles failed too ;)

Later, in 2017, NHTSA denied a request that they require AEB on all cars. They did so saying "NHTSA has expanded its New Car Assessment Program (NCAP) so that the NCAP information for a vehicle notes whether the vehicle is equipped with one or more of these technologies. " Yet they mention no specific test procedure required to achieve this equipage mark. They also said:

As the manufacturers respond to NCAP and carry out their commitments, the Agency is continuously monitoring their efforts to assess whether additional steps, including the possibility of a rulemaking to establish a new standard, might be needed in the future to ensure realization of the potential benefits from the full array of automatic emergency braking technologies.

Doesn't sound like they have a standard yet, and they are being very free market about this.

We still need a much more concise reference that NHTSA has a specific validation method before they will list AEB on the NCAP report if we're going to call it "regulation" and "validation" and argue that what Tesla has right now is totally AEB they just haven't been able to get that report to NHTSA yet. Remember, we have the news that Tesla briefed the NHTSA on the changes and based on that, NHTSA removed the AEB mark. If there was a standard validation process, that would not be needed, the only question would be "where is the test report that you have summitted 20 times before for all your other car models and HW/SW changes?"

A report I'd expect Tesla to easily be able to produce if the removal of Radar was a long planned event that underwent a lot of testing, validation, and quality assurance.
 

powertoold

Active Member
Oct 10, 2014
3,021
6,506
USA
We still need a much more concise reference that NHTSA has a specific validation method before they will list AEB on the NCAP report if we're going to call it "regulation" and "validation" and argue that what Tesla has right now is totally AEB they just haven't been able to get that report to NHTSA yet.

It's not clear, but it seems "common sense" that NHTSA evaluates the AEB in some way. It's also very obvious that Tesla has some form of FCW in the vision-only cars. So the fact that NHTSA has a "NO" on FCW means 1) they haven't tested it or 2) Tesla hasn't had a chance to submit the change for vision-only cars.

I also found this:


11. PASS/FAIL CRITERIA FOR THE PERFORMANCE TESTS​

The Alliance, Honda, AGA and Ford said that the determination that AEB technologies will pass each of the tests in the test procedure seven out of eight times should be changed to be consistent with the five passes out of seven trials that is specified by the NCAP forward collision warning (FCW) test procedures. The Alliance and Ford noted that the agency did not provide data to support the seven out of eight criterion approach. Ford presented the results of a coin toss experiment, which it said indicated that the five out of seven criteria covers 93.8 percent of all possible outcomes, a level whose robustness compares favorably to the 99.6 percent of all possible outcomes covered by the seven out of eight criterion.

Tesla said the planned test procedures include too many tests.

NHTSA notes that for the FCW NCAP, the vehicle must pass five out of seven trials of a specific test scenario, to pass that scenario. The vehicle must pass all scenarios to be recommended.

The agency believes the current FCW test procedure criterion of passing five out of seven tests has successfully discriminated between functional systems versus non-functional systems. Allowing two failures out of seven attempts affords some flexibility in including emerging technologies into the NCAP program. For example, NHTSA test laboratories have experienced unpredictable vehicle responses, due to the vehicle algorithm designs, rather than the test protocol. Test laboratories have seen systems that improve their performance with use, systems degrading and shutting down when they do not see other cars, and systems failing to re-activate if the vehicle is not cycled through an ignition cycle.
 
Last edited:
  • Like
Reactions: rxlawdude

Jeff N

Active Member
Oct 31, 2011
2,461
3,557
Can anyone point me to the law/regulation/policy that says NHTSA won't consider a function to be AEB unless it has passed a specific test, and what that specific test is? It would be interesting to see what Tesla is unable to currently prove they do, which they were able to prove before.
As best as so can tell, there aren’t any standard US government test procedures for the various ADAS (Advanced Driver Assistance System) features but it seems as if they are working on them.

For example, this seems to be a draft of test regulations for pedestrian automatic emergency braking:

 

powertoold

Active Member
Oct 10, 2014
3,021
6,506
USA
for pedestrian automatic emergency braking

Pedestrian AEB isn't one of the boxes on their safety ratings though. Check my NCAP "Final decision" link above.

Screen Shot 2021-05-27 at 9.49.32 PM.png


V. Conclusion​

For all the reasons stated above, we believe that it is appropriate to update NCAP to include crash imminent braking and dynamic brake support systems as Recommended Advanced Technologies.

Starting with Model Year 2018 vehicles, we will include AEB systems as a recommended technology and test such systems.
 

stopcrazypp

Well-Known Member
Dec 8, 2007
10,969
6,073
Can anyone point me to the law/regulation/policy that says NHTSA won't consider a function to be AEB unless it has passed a specific test, and what that specific test is? It would be interesting to see what Tesla is unable to currently prove they do, which they were able to prove before.
Apparently, there are test procedures. This is what I could find in my cursory search:


"tested with the agency’s August 2014 draft crash‐imminent braking (CIB) and dynamic brake support (DBS) test procedures."
That test procedure had been further refined since then and Tesla even commented on it! There is a 200 brake burnishing stop procedure (that many disagreed with and was not in line with Euro NCAP which only had 13), and a steel trench plate based false positive test. More details in the final draft after taking into account all comments in the comment period.

This is the final test procedure that was approved in 2015 and went into effect for the 2018 MY:
"These changes to the New Car Assessment Program are effective for the 2018 Model Year vehicles."
New Car Assessment Program (NCAP)

Even if you want to argue about AEB, the LDW test had been established even earlier than that:
https://safercar.dr.del1.nhtsa.gov/staticfiles/safercar/NCAP/LDW_LKS_2-7-2013.pdf
The FCW test is here:
https://safercar.dr.del1.nhtsa.gov/staticfiles/safercar/NCAP/FCW_NCAP_Test_Procedure_2-7-2013.pdf
https://safercar.dr.del1.nhtsa.gov/NCAP_Test_Procedures
 
Last edited:

gearchruncher

Active Member
Sep 20, 2016
2,510
3,316
Seattle, WA
It's not clear, but it seems "common sense" that NHTSA evaluates the AEB in some way. It's also very obvious that Tesla has some form of FCW in the vision-only cars. So the fact that NHTSA has a "NO" on FCW means 1) they haven't tested it or 2) Tesla hasn't had a chance to submit the change for vision-only cars.

It's fascinating that Tesla has managed to release the 3, HW3 (the 3 shipped with HW2.5), and the Y, plus a bunch of SW changes all without losing AEB ratings. Seems like they know the process pretty well.

But with this change, they suddenly don't have enough data, and now NHTSA, Consumer Reports. and IIHS are publicly saying the 3/Y are dropping in their ratings, and there are hundreds of news articles now about this.

All because Tesla hasn't had a "chance" to submit the change for this well planned transition to vision only.
 

stopcrazypp

Well-Known Member
Dec 8, 2007
10,969
6,073
It's fascinating that Tesla has managed to release the 3, HW3 (the 3 shipped with HW2.5), and the Y, plus a bunch of SW changes all without losing AEB ratings. Seems like they know the process pretty well.

But with this change, they suddenly don't have enough data, and now NHTSA, Consumer Reports. and IIHS are publicly saying the 3/Y are dropping in their ratings, and there are hundreds of news articles now about this.

All because Tesla hasn't had a "chance" to submit the change for this well planned transition to vision only.
NHTSA did make a comment to the media that it "only includes check marks for the model production range for the vehicles tested". People parsed that to mean that it might just be NHTSA wants to test this change before giving the check marks back.

For the release of the 3 and Y, NHTSA necessarily had to do separate tests for them, so Tesla wouldn't have "lost" AEB ratings, rather they would have been "established" as part of their release. For HW3, Tesla didn't indicate they made any changes to AEB/FCW. As for software updates, other than the updates related to AP2 and AP2.5 (which came before NHTSA officially started evaluating AEB for NCAP in 2018 MY), I didn't find any updates related to that (at least wasn't significant to mention in release notes).

Car Safety Ratings | Vehicles, Car Seats, Tires | NHTSA
You can see by searching for Tesla in the safety rating, the 2018 Tesla Model X and 2018 Model S were the first to have both CIB and DBS checkmarks:
Vehicle Detail Search | NHTSA
Vehicle Detail Search | NHTSA
The 2018 and 2019 Model 3 actually was rated not to have DBS:
Vehicle Detail Search | NHTSA
Vehicle Detail Search | NHTSA
Model 3 got it in 2020:
Vehicle Detail Search | NHTSA
That seems to suggest NHTSA does not update the rating related to software changes (meaning they do it based on testing/information at the time, and is not updated later on based on Tesla's software updates).

It's also not unprecedented for NHTSA to give a different rating for "early release" and "later release". Example with FCW/LCW removed from "later release" Model S/X related to the AP2.0 change:
Vehicle Detail Search | NHTSA
Vehicle Detail Search | NHTSA

So we have precedence already of this with Tesla (Consumer Reports's Jake Fisher mentioned the same), and it's not as you seem to suggest that this is the first time for Tesla.

That history seems to also suggest we might not see NHTSA give an updated rating until perhaps the next model year, even if Tesla gets the software updated this year. If this is true, it also makes sense of why they might do this only on the Model 3/Y first, as those are their highest volume vehicles, so if they have to pay for the testing of a mid-year change out of pocket, those would be the most economical to do so (as costs are spread out to most volume). That suggests Tesla may wait out the next model year to do it for lower volume vehicles like S/X to have the tests as part of that model year change as a whole.

As for removal by IIHS and Consumer Reports, they said explicitly they are just following NHTSA's move. IIHS is planning to retest the vehicle after that change before updating their rating (so same situation, they just haven't done the testing).
Consumer Reports drops 'top pick' rating for Tesla Model 3
 
Last edited:
  • Like
Reactions: powertoold

hamzah1024

Member
Apr 29, 2021
48
45
Austin, TX
According to this article Tesla vehicles without radar still have Autopilot safety features despite what the media is saying - Electrek , all cars currently being delivered have all of the safety features (which includes AEB) and the NHTSA will be testing this next week.
Teslas blog post said "Emergency Lane Departure Avoidance may be disabled at delivery." Emphasis theirs.

What some idiots on the internet, and by extension, the media, took that to mean Automatic EMERGENCY BRAKING (Different from lane departure) was removed/disabled. How they made that leap, I dont know.

Tesla even goes further and says "All other available Autopilot and Full Self-Driving features will be active at delivery, depending on order configuration." Emphasis mine. Meaning, they arent cutting out anything not mentioned above. Unless you are willing to believe that Tesla would blatantly lie about the inclusion of safety features, you have to conclude that everything but Lane departure avoidance is still there.
 

Products we're discussing on TMC...

About Us

Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.

Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


SUPPORT TMC
Top