TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

Seeing the world in autopilot, part deux

Discussion in 'Autonomous Vehicles' started by verygreen, Sep 25, 2018.

  1. Bladerskb

    Bladerskb Senior Software Engineer

    Joined:
    Oct 24, 2016
    Messages:
    1,674
    Location:
    Michigan
    Great analysis as alway.

    Here's a mobileye comparison with 3d bounding box. That and the amount of jitters and unstable detection. Looks like Tesla is still very much behind in object detection.
     
    • Disagree x 4
    • Informative x 1
    • Like x 1
  2. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,383
    Location:
    TN
    so what's the importance of the 3D bounding box @Bladerskb ? you just need to know where you cannot drive on your side of the obstacle, who cares how is it shaped on the other sides anyway (in other words you can skin this cat in other ways). and they do this with driveable space, the bounding boxes are just for information more or less and it's possible though unlikely) that they can do 3D bounding boxes, there's depth information, just no orientation vector to draw the 3d part.

    Show us 3D planned path prediction or driveable space prediction from other vendors if you have the samples? Those seem to be pretty important things.
     
    • Like x 8
  3. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,383
    Location:
    TN
    oh boy do you have a surprise waiting! It looks like there are no roundabouts in California ;)
     
    • Funny x 5
    • Like x 1
  4. Muzzman1

    Muzzman1 Member

    Joined:
    Feb 8, 2014
    Messages:
    478
    Location:
    Los Angeles
    Amazing work @verygreen and your team! Just awesome!
     
    • Like x 1
  5. GreenT

    GreenT Member

    Joined:
    Mar 14, 2014
    Messages:
    768
    Location:
    Florida / France
    Is this conference open to anyone in Paris?
     
    • Funny x 1
  6. daktari

    daktari Member

    Joined:
    Jan 21, 2017
    Messages:
    667
    Location:
    Norway
    Looking forward to see roundabouts!

    And I am worried about "sensitivity and specificity" seeing this.
     
  7. Bladerskb

    Bladerskb Senior Software Engineer

    Joined:
    Oct 24, 2016
    Messages:
    1,674
    Location:
    Michigan
    This will be a rehash from reddit but i will try to update with greater details.

    Actually 3D bounding box might be the most important thing in object detection. A 2d bounding box (which includes shapes) doesn't include important information which you need. For example the precise orientation of a car, with that you can predict its actions. This is vital in a dense environment, during parking, in a parking lot. Is this car trying to pull out or not. What precise direction is he pulling to, With just the information necessary to produce 2d bounding box, you don't see it. But the information is reflected in the information necessary to produce a 3d bounding box. Information like, is this the left side of a car, the door might pop out, is the door currently open? is this car trying to complete a turn. is this car coming at me or... the list goes on. As Mobileye Amon says. 2d bounding box is irrelevant.

    While its impressive that distances and speed are done by visual. Mobileye has been doing this since eyeq3. There's no tangible difference between atleast the firmware you have unraveled versus what eyeq3 has been doing since 2014 production date.

    I'm just laying out that the lack of 3d boxes and the instability of the detection shows their weakness compared to mobileye.


    3d bounding box whether you are using lider or camera gives you the precise orientation of an object with which you can deduce its precise moment to moment decision. This is very important in dense traffic or parking lot.

    Imagine if you painted inside of the 2d boxes with solid color and then look at it in the perspective as a human. Notice how you lose all indication of what's going on in the scene, only that there are objects in front of you but no idea the scenario, orientation or state they are in.

    oh wait, you already did that.




    Notice how you don't know if the car infront of you is turning and where they are turning to or not. Or even if they are going the same direction as you. you're basically driving blind.

    This is even now more evident when you are driving on dense surface streets. A car could be in-front of you at a intersection in the adjacent turning lane, preparing to turn into the parallel lane next to you. With a 2d bounding box info, you won't know if that car is driving forward parallel to you or driving forward adjacent to you or making a turn.

    Ask yourself, can you drive in dense traffic with the above view superimposed to your view? Answer is absolutely not. You need to know the orientation of cars.


    If only driving were just about not hitting things and not prediction, knowing and planning around the actions or perceived actions of another driver.
    Depth info by itself won't be useful, like a raw lidar data, depth info is simply just pixels and raw lidar data is simply a bunch of cloud points. Its when you process the info to produce quantifiable output that matters and based on your statements those outputs currently doesn't exist.

    Will provide more in another post.
     
    • Helpful x 1
    • Informative x 1
  8. Snuffysasa

    Snuffysasa Member

    Joined:
    Feb 24, 2017
    Messages:
    267
    Location:
    Minneapolis
    3D bounding boxes = object boundaries?

    Why do you say 3d bounding boxes are gimmick, but you need object boundaries?

    I guess I am agreeing with @Bladerskb here. You do need 3d bounding boxes,

    But I don't think object height is that important.

    But you definitely need 3d object position along with orientation and width/length in order to get object boundaries.


    In this case Tesla has 2D box with depth, and if that depth is accurate, then they could make crude 3d object boundaries with that data... however in many cases they will end up assuming an object is larger than it actually is, which is probably not a big deal for now
     
  9. Vitold

    Vitold Active Member

    Joined:
    Aug 10, 2015
    Messages:
    1,692
    Location:
    NM
    #29 Vitold, Sep 25, 2018
    Last edited: Sep 25, 2018
    Given that car’s body follows front wheels AI, in dense environment, would be better off following orientation of the front wheel rather than the whole body. For example, when parked car wants to merge with traffic, it’s possible velocity cannot be predicted by its 3D bounding box.
     
    • Like x 1
  10. lunitiks

    lunitiks Cool James & Black Teacher

    Joined:
    Nov 19, 2016
    Messages:
    2,708
    Location:
    Prawn Island, VC
    The "toy car" orientations in IC sort of indicate that there might be some 3D understanding going on of the diiferent "vechiles" :) No?
     
  11. mongo

    mongo Well-Known Member

    Joined:
    May 3, 2017
    Messages:
    9,201
    Location:
    Michigan
    On winter roads, trajectory trumps orientation...
     
    • Funny x 4
  12. J1mbo

    J1mbo Active Member

    Joined:
    Aug 20, 2013
    Messages:
    1,412
    Location:
    UK
    Given the data we can see in the visualisation, path prediction of the other vehicles based on historical movement would seem to be pretty straightforward IMO.
     
  13. I.a.n

    I.a.n Member

    Joined:
    Dec 6, 2017
    Messages:
    18
    Location:
    England
    Thanks - its a really interesting insight to how much further the internals are than what we see.

    Does this exist port on the non-dev versions? Do you think plugging in an ssd drive will be necessary for the V9 dash-cam feature or is there sufficient high-speed storage for there to be a usable dash cam without? (I think a dash cam with a rolling 30 second buffer that is saved if airbags deploy is better than nothing, but I'd probably keep my existing dash cam if that is the limit of the functionality).

    How much of the metadata does this represent? I.e. was there enough uninterpretable metadata for street signs and road markings such as give way lines & pedestrian crossings to be being recognised, or is this something that will likely need the AP3 hardware?
     
    • Helpful x 1
  14. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    3,583
    Location:
    Bay Area
    Can’t tell if you’re joking or not :)
     
  15. I.a.n

    I.a.n Member

    Joined:
    Dec 6, 2017
    Messages:
    18
    Location:
    England
    The new nav miscounts roundabout exits (i.e. it will often say "take the first exit" when it means, and shows, the second exit on the screen) - for about 1 in every 20 roundabouts near me in the UK. We'd (some UK Tesla owners) speculated this was due a relative lack of testing due to a lack roundabouts near Tesla HQ. The comments above (joking or not!), made me wonder - a little searching found Mapping America's Resistance to Traffic Roundabouts - CityLab which has some interesting stats.

    The US has about 10x fewer roundabouts per junction than the UK, so the roundabout exit-count error would be 10x less likely to happen. But even in the US, the roundabouts are not evenly spread. California actually had the 2nd highest number of roundabouts per state, but the author states "Where I live now in California there are literally no roundabouts" (presumably just his local area given the other statistic).
     
  16. GreenT

    GreenT Member

    Joined:
    Mar 14, 2014
    Messages:
    768
    Location:
    Florida / France
    I would hazard your number is way off. More like 1 roundabout for every 100,000 in the UK.
    (Metrics are covered in that article, thanks.)
    The UK itself IS a roundabout. :)
    There are several in SWFL (Southwest Florida) (more than I have ever seen for all the places I've lived in the US) and yes, I've noticed the exit count is off.
     
  17. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    #37 AnxietyRanger, Sep 25, 2018
    Last edited: Sep 25, 2018
    Great work @verygreen and @DamianXVI - the whole Tesla enthusiast community is indebted to you!

    One thing occurred to me just today.

    I'm running 2018.32.4 040c866 and I have noticed its car recognition as having improved compared to older versions. It actually seems to show pretty accurately in-lane and next-lane cars on the IC, although of course it does not separate between vehicle type like AP1 does. But it is pretty fluid and accurate IMO.

    One thing that is special is that it clearly uses the fisheye (and/or radar) to see a wider angle than AP1-like narrow angle would be capable of. I was driving, in traffic, on a winding road and it actually displayed the queue of cars in front of me, that spanned a significant portion of my FoV towards the left - with the lane turning so sharply, the cars spanned a range much wider than what the main/narrow cameras on AP2 (or the AP1 camera) could see... So far so good, pretty impressive actually.

    But here comes my point: It displayed all those cars in a correct arc, in their correct places, but it was unable to recognize the lane's angle, so they were "hanging in air". Without the lane approximation or movement to guide the IC's display system, and clearly lacking or ignoring any 3D data that might (or might not) exist, it simply defaulted to all the cars facing my driving direction. So the queue of stopped cars - winding/queueing towards the left in reality - ended up looking like a bunch of cars (vertically) parked next to each other in a left-winding arc.

    3D boxes would help with such displays - and of course with any decision-making that might rely on the direction and angle of other cars.
    I've always thought it basically just guesses based on the lane angle (which in itself is very much only an approximation on the IC), though I admit the "toy cars" sometimes flipflop directions on the IC, which might suggest it is trying to do more than that... I guess it might also use their movement as a guide, which wasn't available with a queue stopped in traffic, but that doesn't seem very reliable IMO.

    Please keep up the good research and discussion, guys! Thank you.
     
    • Helpful x 1
  18. Darmie

    Darmie Supporting Member

    Joined:
    Jan 13, 2016
    Messages:
    627
    Location:
    Clear Lake TX. (Currently in lake Charles LA)
    Great stuff. Very interesting to see the Advance stage Tesla is in on FSD.
     
  19. Pale_Rider

    Pale_Rider Member

    Joined:
    Jul 28, 2016
    Messages:
    448
    Location:
    Houston, TX
    Seeing a local neighborhood or two way unmarked street would be interesting to see if it was beginning to understand a drivable path in that environment.
     
  20. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,383
    Location:
    TN
    Already over.
     
    • Informative x 1

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC