TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

IIHS compares driver assistance Tesla, MB, BMW Volvo

Discussion in 'Autonomous Vehicles' started by bhzmark, Aug 7, 2018.

  1. bhzmark

    bhzmark Supporting Member

    Joined:
    Jul 21, 2013
    Messages:
    2,447
    Tests uncover issues for advanced features

    Some interesting highlights are below the Model S tested is a 2016 with 7.1 software and probably AP1 hardware. It did well except for hill crests. The Model 3 with AP2 did even better.

    upload_2018-8-7_6-24-41.png


    The 2017 BMW 5-series with "Driving Assistant Plus," 2017 Mercedes-Benz E-Class with "Drive Pilot," 2018 Tesla Model 3 and 2016 Model S with "Autopilot" (software versions 8.1 and 7.1, respectively) and 2018 Volvo S90 with "Pilot Assist" were evaluated. All five have automatic emergency braking systems rated superior by IIHS.

    One series involved driving at 31 mph toward a stationary vehicle target with ACC off and autobrake turned on to evaluate autobrake performance. Only the two Teslas hit the stationary target in this test.

    The same test was repeated with ACC engaged and set to close, middle and far following distance in multiple runs.

    With ACC active, the 5-series, E-Class, Model 3 and Model S braked earlier and gentler than with emergency braking and still avoided the target. The cars slowed with relatively gradual decelerations of 0.2-0.3 gs, braking in the same manner no matter the distance setting. Braking before impact was earlier for the Teslas than for the 5 series and E-Class.

    . . . .

    A fourth scenario involved the test vehicle following a lead vehicle, which then changed lanes to reveal a stationary inflatable target vehicle in the path ahead when the time to collision was about 4.3 seconds.

    None of the vehicles crashed into the target, and the 5 series, E-Class and Teslas all braked earlier and gentler than the S90, similar to the active ACC test.

    . . .Out on the road, engineers noted instances in which each vehicle except the Model 3 failed to respond to stopped vehicles ahead.

    . . .

    Unnecessary or overly cautious braking is an issue IIHS noted in the Model 3. In 180 miles, the car unexpectedly slowed down 12 times, seven of which coincided with tree shadows on the road. The others were for oncoming vehicles in another lane or vehicles crossing the road far ahead.

    . . .

    Only the Model 3 stayed within the lane on all 18 trials. The Model S [with the older hardware and software] was similar but overcorrected on one curve, causing it to cross the line on the inside of the curve in one trial. None of the other systems tested provided enough steering input on their own to consistently stay in their lane, often requiring the driver to provide additional steering to successfully navigate the curve.

    The E-Class stayed within the lane in 9 of 17 runs and strayed to the lane marker in five trials. The system disengaged itself in one trial and crossed the line in two. The 5 series stayed within the lane in 3 of 16 trials and was more likely to disengage than steer outside the lane. The S90 stayed in the lane in 9 of 17 runs and crossed the lane line in eight runs.

    . . .

    The E-Class stayed in its lane in 15 of 18 trials and on the line in one trial, continuously providing steering support without erratic moves when lane lines weren't visible. The Model 3 also stayed in the lane in all but one trial, when it hugged the line.

    In contrast, the 5-series, Model S and S90 struggled. The 5-series steered toward or across the lane line regularly, requiring drivers to override the steering support to get it back on track. Sometimes the car disengaged steering assistance on its own. The car failed to stay in the lane on all 14 valid trials.

    The Model S was errant in the hill tests, staying in the lane in 5 of 18 trials. When cresting hills, the Model S swerved left and right until it determined the correct place in the lane, jolting test drivers. It rarely warned them to take over as it hunted for the lane center. The car regularly veered into the adjacent lanes or onto the shoulder.
     
    • Informative x 6
  2. bhzmark

    bhzmark Supporting Member

    Joined:
    Jul 21, 2013
    Messages:
    2,447
    And another article released today:

    Advanced features reduce Model S claims

    Some highlights:

    The combined crash avoidance features on the Tesla Model S are reducing third-party physical damage and injury liability claims, while the benefit of adding "Autopilot" is limited to lowering collision claims.

    . . .
    The combined driver assistance features on the 2014–16 Model S lowered the frequency of claims filed under property damage liability (PDL) coverage by 11 percent and the frequency of claims under bodily injury (BI) liability coverage by 21 percent, compared with the 2012–14 Model S without the technology, HLDI found.

    . . .

    In this limited analysis, HLDI found that the frequency of claims filed under PDL, BI, MedPay and PIP didn't change once Autopilot was enabled, but the frequency of collision claims fell by 13 percent.

    "To get a better picture of how Autopilot is affecting claims, we need more data on how many Teslas are equipped with Autopilot and how often it is used," says Matt Moore, HLDI's senior vice president. "The reductions in the frequency of third-party physical damage and injury liability claims associated with Tesla's version 1 hardware are in line with the benefits HLDI has documented for comparable systems from other manufacturers."

    Moore adds, "When we evaluated Teslas with the version 1 hardware after the Autopilot software was deployed, we saw a significant reduction in collision claim frequency but no other changes."
     
  3. Frizull

    Frizull Member

    Joined:
    Nov 28, 2016
    Messages:
    13
    Location:
    Geneva, Switzerland
    It's a mystery to me as to why some of those agencies continue to test against older software versions. Publicly available data shows the Tesla fleet has incredible update adoption. Those hill lane-holding imperfections have been fixed for months now.
     
    • Informative x 1
    • Like x 1
  4. Saghost

    Saghost Well-Known Member

    Joined:
    Oct 9, 2013
    Messages:
    5,552
    Location:
    Delaware
    "We're not ready to say which company has the best level 2 driver assistance" while the Model 3 turns in a basically perfect score and nothing else comes close. Charming.

    (Driver assistance, not safety features. Another takeaway from this test is that Tesla needs to work on emergency braking with ACC off, which shouldn't really surprise folks that have been reading here. The test also shows how far Tesla has come in the last couple years.)
     
    • Like x 2
  5. jaguar36

    jaguar36 Active Member

    Joined:
    Apr 10, 2014
    Messages:
    1,762
    Location:
    NJ
    IIHS still doesn't seem to get it. They don't even mention what version of the software its using beyond 7.1 and 8.1.
     
    • Helpful x 1
    • Like x 1
  6. Spidy

    Spidy Active Member

    Joined:
    Feb 7, 2015
    Messages:
    1,299
    Location:
    EU
    Except...

    Unnecessary or overly cautious braking is an issue IIHS noted in the Model 3. In 180 miles, the car unexpectedly slowed down 12 times, seven of which coincided with tree shadows on the road. The others were for oncoming vehicles in another lane or vehicles crossing the road far ahead.
     
    • Helpful x 1
  7. Saghost

    Saghost Well-Known Member

    Joined:
    Oct 9, 2013
    Messages:
    5,552
    Location:
    Delaware
    A valid point. I was more focused on the chart at the end, which didn't include that part. Phantom braking has certainly annoyed a lot of people here.
     
  8. jaguar36

    jaguar36 Active Member

    Joined:
    Apr 10, 2014
    Messages:
    1,762
    Location:
    NJ
    Its also been improved upon substantially in recent releases, however since IIHS doesn't seem to understand how OTAs work, they don't even acknowledge this as a possibility.
     
  9. croman

    croman Active Member

    Joined:
    Nov 21, 2016
    Messages:
    3,001
    Location:
    Chicago, IL
    Its not that the IIHS doesn't understand OTA or software revisions, its that they want meaningful significant data and if the Teslas keep changing, they cannot build that data. Clearly Tesla's approach is superior and having one car with 7.1 and another with 8.1 shows that iterative improvement.
     
  10. Saghost

    Saghost Well-Known Member

    Joined:
    Oct 9, 2013
    Messages:
    5,552
    Location:
    Delaware
    But as Mr Jaguar notes, they didn't provide a build version. We've been on 8.1 for more than a year, and seen big changes to AP performance during that time all with the 8.1 version number.

    (Admittedly, this is Tesla making things confusing, but the only way I see that IIHS can unconfuse it is to show us the build our date of their version.)
     
    • Like x 2
  11. Trent Eady

    Trent Eady Non-Member

    Joined:
    May 10, 2017
    Messages:
    183
    Location:
    Whiterun, Skyrim
    This is potentially concerning:

    “Looking at first-party injury coverage types, HLDI found a 29 percent increase in the frequency of claims under medical payment (Medpay) coverage and a 39 percent increase in the frequency of personal injury protection (PIP) claims.

    MedPay covers injuries to an at-fault driver or passengers in that driver's vehicle, while PIP coverage is sold in states with no-fault insurance systems. This coverage pays for injuries to occupants of the insured vehicle, no matter who is at fault.”

    But it’s hard to know what to do with the IIHS data — either the good or the bad. If you look at their report, a lot of the error bars are much larger than the effect sizes. What the heck are supposed to do with this?

    [​IMG]
     
    • Informative x 1
  12. Bladerskb

    Bladerskb Like how many times do i have to be right?

    Joined:
    Oct 24, 2016
    Messages:
    859
    Location:
    Michigan
    One thing to note is that GM Supercruise was not included in this test. which has been independently shown to be the best level 2 driver assistance.
     
  13. EinSV

    EinSV Active Member

    Joined:
    Feb 6, 2016
    Messages:
    3,118
    Location:
    NorCal
    #13 EinSV, Aug 11, 2018 at 3:14 PM
    Last edited: Aug 11, 2018 at 3:40 PM
    fw 8.0 was initially released in September 2016 (before AP2) so the 2016 Model S with 7.1 is almost certainly AP1.
    Tesla

    Bizarre that the IIHS would test using two year old firmware even on an AP1 car.

    IMO, it was very misleading of the IIHS not to mention in their article that the Model S they tested was an AP1 car with long outdated, two-year old firmware. The ordinary reader would not have a clue what firmware 7.1 is.
     
    • Like x 2
  14. SlicedBr3ad

    SlicedBr3ad Member

    Joined:
    Feb 25, 2017
    Messages:
    786
    Location:
    Fl
    It's amazing that they still had a car with such old version. It's possible that even they were clueless. [Sigh]
     
  15. EinSV

    EinSV Active Member

    Joined:
    Feb 6, 2016
    Messages:
    3,118
    Location:
    NorCal
    Here's a summary of the lane-keeping results from the last column of the chart above (dropping the Model S with outdated firmware):

    Successfully navigated and stayed within lane:

    Car Curves Hills
    Model 3 100% 94.4%*
    Mercedes E 50% 83.3%
    Volvo S90 50% 50%
    BMW 16% 0%

    *Model 3 touched but did not cross dividing line on one occasion.
     
    • Informative x 2
  16. croman

    croman Active Member

    Joined:
    Nov 21, 2016
    Messages:
    3,001
    Location:
    Chicago, IL
    It wouldn't have worked on their test roads. LOL, such a great system!
     
    • Like x 1
  17. Bladerskb

    Bladerskb Like how many times do i have to be right?

    Joined:
    Oct 24, 2016
    Messages:
    859
    Location:
    Michigan
    #17 Bladerskb, Aug 12, 2018 at 6:06 PM
    Last edited: Aug 12, 2018 at 6:17 PM
    objectively better than Tesla AP though..
    We tested out Tesla Autopilot and Cadillac's Super Cruise — here's which one we liked better
    The Battle for Best Semi-Autonomous System: Tesla Autopilot Vs. GM SuperCruise, Head-to-Head
    Cadillac's semi-autonomous system is better than Tesla's or Volvo's
    Cadillac Super Cruise Review: I like this more than Tesla Autopilot
    How good is Cadillac’s Super Cruise ‘hands-free’ car technology?

    and get this. GM released an update for super-cruise in June that improved three things:

    “GM seems to have heard some of the criticisms we had of Super Cruise, because this update fixes several of those,” he says.

    The updates include:

    • It’s easier for drivers to turn on Super Cruise. We found that the symbol that notifies drivers when using Super Cruise is allowed shows up more often, especially after lane changes. Previously, it could be difficult and frustrating for our drivers to determine why the symbol wasn’t appearing when all other factors seem to suggest it should be allowed.

    • The adaptive cruise control system has been tweaked to provide smoother acceleration and braking.

    • GM added 15 new messages to give drivers more information about why Super Cruise is not available or why it shut off. Those include messages alerting drivers that ACC needs to be on to start Super Cruise; that it can’t be used if forward-collision warning isn’t turned on; that the road they’re on isn’t appropriate for the system; or that it simply can’t find lane lines to follow.
    GM Makes Super Cruise Driver-Assist System Easier to Use

    So all the complaints have been addressed, easier and quicker to activate. ACC/cut-in greatly improved. And more UI/UX upgrades.

    The reason i'm so pro supercruise is just not because its the best L2 system, completely hands-free with no nags but the fact that its the only system that gives you a visual feedback, hap-tic feedback (vibrating seat), and audio message when it disengages.

    Its really a no contest right now.

    Here's a map of all supercruise mapped highways.

    Cadillac Super Cruise
     
  18. Bladerskb

    Bladerskb Like how many times do i have to be right?

    Joined:
    Oct 24, 2016
    Messages:
    859
    Location:
    Michigan
    also

    "The updated Super Cruise software did add an enhancement for control around exit ramps"
     
  19. Randy Spencer

    Randy Spencer Member

    Joined:
    Mar 31, 2016
    Messages:
    736
    Location:
    Alameda, CA
    Crazy Cadillac fanboy cannot get anyone at a Cadillac site to listen to him? Sure am glad I have a Tesla that lets me activate auto pilot after leaving my driveway. I have that second set of eyes on my driving almost EVERYWHERE I go. No nags if I rest my hand on the wheel.

    I really was motivated to get the Tesla to take care of our weekly drive to our cabin in the Sierras. Supercruise don't go there. Auto Pilot does 99% of that drive with a single disengagement and has improved over the months we have had the car with better and better driving. New updates every week or two.

    -Randy
     
    • Like x 2

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC