TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

It's Not a Bug. It's a Feature.

Discussion in 'Autopilot & Autonomous/FSD' started by NerdUno, Apr 4, 2019.

  1. NerdUno

    NerdUno Member

    Joined:
    Dec 18, 2016
    Messages:
    652
    Location:
    Charleston, SC
  2. Tam

    Tam Well-Known Member

    Joined:
    Nov 25, 2012
    Messages:
    8,101
    Location:
    Visalia, CA
    This feature can be very helpful if a good "hacker" would put some stickers on the road to make sure the car would steer away from and not slam right into a median concrete wall.

     
    • Informative x 2
    • Disagree x 2
    • Like x 1
  3. favo

    favo P3D+ owner

    Joined:
    Apr 5, 2012
    Messages:
    1,020
    Location:
    Durham, NC
    This doesn't seem like that big a deal. They are using year-old firmware version (2018.6.1). Fooling auto-wipers with a big TV image--who cares? Controlling steering with a gamepad in the car--Tesla says they fixed relevant bugs in 2017 and 2018, also requires non-trivial physical access to car so it can be rooted, enabling the attacker to inject bogus data into the CAN bus. If you have physical access, lots of bad things can be done.

    Using the three stickers to fool the car into changing to a fake lane (supposedly going the "wrong" way) is slightly more worrying, except that they didn't demonstrate this on a real road, so there was no map data that might have helped the car figure out where the real lanes are and which way traffic flows. Also, this relied on the lack of other lane markings in close proximity to the bogus markings, limiting the attack scenario to intersections (and maybe poorly marked lanes) as they mentioned. Perhaps it would have still screwed up even with good map data or good surrounding lanes lines, but we don't know. Also, you aren't supposed to use AutoPilot in these scenarios yet, so user error. Plus, an alert driver would take over right away.

    As anecdotal "evidence," there is ongoing construction on a highway near me, where the lanes have been shifted, and there are lots of confusing, partially erased lane markings. In particular, there are three or four very clear old lane markings going diagonally across a lane next to a Jersey barrier. EAP has handled this every time I've gone through there (~20 times), except for the first time, 6+ months ago.
     
  4. 1375mlm

    1375mlm Member

    Joined:
    Jun 29, 2017
    Messages:
    197
    Location:
    Texas, Houston
    I don’t think that car was on AP. First sign is it “drifting” gently. When AP recognizes a lane widening or narrowing, like when merging, it will close that gap quick. There’s no gradual drifting over to center for that. And ultrasonic sensors would have seen that coming way before, and stayed away from the divider.
    Seems quite clear the car was not on AP.
     
    • Informative x 1
    • Disagree x 1
  5. MikeS1

    MikeS1 Member

    Joined:
    Jan 20, 2019
    Messages:
    53
    Location:
    Calgary
    Autopilot was not on, Tesla I believe confirmed this and its quite obvious in the video it was not on.
     
    • Like x 2
    • Informative x 1
    • Disagree x 1
  6. Tam

    Tam Well-Known Member

    Joined:
    Nov 25, 2012
    Messages:
    8,101
    Location:
    Visalia, CA
    How about in a case that Tesla confirmed?

    What We Know About Last Week's Accident

    If hackers can do such a simple Autopilot redirection with stickers maybe the Apple Engineer didn't have to die.
     
  7. electronblue

    electronblue Active Member

    Joined:
    Oct 1, 2018
    Messages:
    2,325
    Location:
    Earth
    Source?

    Because it sure looks like to me the car got confused by the diamond marking being the right lane marker and trying to cross between the left lane marker and diamond... thus ending up losing sight of the real right lane marker and then repositioning itself between the curb as the left lane marker and the actual left lane marker as the right lane marker... and then simply running out of space.

    Can be a coincidence of course but given the sticker hack looks like a very potential reason too.

    [​IMG]
     
  8. electronblue

    electronblue Active Member

    Joined:
    Oct 1, 2018
    Messages:
    2,325
    Location:
    Earth
    #8 electronblue, Apr 5, 2019
    Last edited: Apr 5, 2019
    But with great sensor fusion such as Lidars or even perhaps more radars (side-facing etc) things like concrete barriers could be detected before vision gets there and thus could be avoided at all costs. Ultrasonics are not reliable at highway speeds...

    Tesla has a very limited sensor suite compared to what many others are developing outside of vision so Tesla is very reliant on being able to visually detect things — and obviously that part is still failing a lot of the time.
     
    • Like x 1
  9. favo

    favo P3D+ owner

    Joined:
    Apr 5, 2012
    Messages:
    1,020
    Location:
    Durham, NC
    Not sure how that would have mattered in this case. They were not testing on a real road with curbs or barriers or traffic lights, etc. It looked like a runway or parking lot where they had painted some lines. What they showed was that if all you have to go on is some intermittent lanes lines, it's not that hard to fool it by adding your own misleading ones.
     
  10. electronblue

    electronblue Active Member

    Joined:
    Oct 1, 2018
    Messages:
    2,325
    Location:
    Earth
    Yes but you never run into a concrete barrier if you can avoid it, lane lines be damned. A side-facing Lidar would have reliably seen the barrier and its distance. Perhaps side-facing radar too.
     
  11. favo

    favo P3D+ owner

    Joined:
    Apr 5, 2012
    Messages:
    1,020
    Location:
    Durham, NC
    All of my comments were related to the Keen testing in the OP. I should also point out that article title was misleading, since there was not actually any traffic, oncoming or otherwise.

    Regarding the video of the grey Model 3 running sideswiping the barrier, I see no evidence they were on AP or even TACC. Original poster of that video said he thought the Tesla might have been trying to race a Chevy, and that both cars had been driving erratically for 15 minutes. Wouldn't be surprised if one or both drivers were impaired.
     
    • Like x 1
  12. emmz0r

    emmz0r Senior Software Engineer

    Joined:
    Jul 12, 2018
    Messages:
    1,123
    Location:
    Norway
    Yeah I chuckled at this. I mean if you need to get into a car and open a bunch of panels, and root it no less, I would think it's easier to demonstrate dangerous situations by just pulling the steering wheel so you land in a ditch :p
     

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC