TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC
  1. TMC is currently READ ONLY.
    Click here for more info.

Neural Networks

Discussion in 'Autopilot & Autonomous/FSD' started by lunitiks, Nov 5, 2017.

  1. ZeApelido

    ZeApelido Active Member

    Joined:
    Jun 1, 2016
    Messages:
    2,641
    Location:
    The Peninsula, CA
    Neat insight. Do you think think this then hints to a NNet eventually controlling end to end driving?
     
  2. Snuffysasa

    Snuffysasa Member

    Joined:
    Feb 24, 2017
    Messages:
    487
    Location:
    Minneapolis
    Thank you @jimmy_d I was waiting for this.


    Excellent! verygreen mentioned something like this too.

    These are for precision localization, which is necessary to use HD maps.

    Excellent point.

    I mean, it I wouldn't say it is recognizing them functionally....

    It is detecting that there IS NOT road there, which I say is different from detecting that there IS an unknown/unclassified object there.


    So this was exactly my first reaction... and I still think it is possible that this is the output of a NN. But then I thought, this path could be the lanes(output from NN) then some very simple algorithm that merges them into a path. Which would explain why they behave that way, if they are a directly correlated to the lane output which is probabilistic with very little processing of them.

    But certainly its very possible that the 2nd layer NN that understands the shape of lanes is also outputting this path.

    I am also not sure AP uses the orange predicted path to steer the vehicle. I think its possible they are testing this probabilistic path in parallel with using some other system for generating and following a trajectory. Or this is purely a visualization/dev tool.

    +1

    +1

    Yes many ADAS systems are using HD maps to improve performance,

    But Every commercial self driving product /service that is planning on making some kind of launch... commercial service or consumer product uses HD maps, and I expect Tesla to do the same as Autopilot matures.

    And the evidence from a few years ago, I am not sure what you are referring too, but if I am thinking of the same thing you are, then I would not call that HD maps, but just regular maps.... (using a rough map and GPS to extract various road attributes and meta data vs using centimeter accurate maps with localization assets for the AV to detect to localize and use then use the map for path planning and even steering the vehicle)

    I am also very curious to see evidence to that, @verygreen has from time to time said he has seen traces of newer HD maps in recent updates, but nothing that appears to be enabled yet.
     
  3. Snuffysasa

    Snuffysasa Member

    Joined:
    Feb 24, 2017
    Messages:
    487
    Location:
    Minneapolis
    I think it suggests the opposite.
     
  4. im.thatoneguy

    im.thatoneguy Member

    Joined:
    Oct 25, 2017
    Messages:
    315
    Location:
    Seattle, WA
    I would say that the lack of evidence is proof that it's not. The yellow drive path appears to be completely constructed by the info being displayed. I would assume that we would see evidence of it using the maps to "cheat" and have premonitions of what is up ahead beyond what we see (at least in the highway demos). Instead it acts 'surprised' by every new twist and turn of the lane.
     
    • Informative x 1
  5. Snuffysasa

    Snuffysasa Member

    Joined:
    Feb 24, 2017
    Messages:
    487
    Location:
    Minneapolis
    Right, verygreen said all the references to maps seem to be not enabled. So it is clear that the current version of 8.1 is not using HD maps
     
  6. strangecosmos

    strangecosmos Non-Member

    Joined:
    May 10, 2017
    Messages:
    1,039
    Location:
    The Prime Material Plane
    Thanks for your analysis, Jimmy!

    Could it be that Autopilot knows the curve of the road ahead — before actually seeing it — because it’s using HD maps? This is one of the intended uses of HD maps: to know what’s up ahead before the car’s sensors see it.
     
  7. Tam

    Tam Well-Known Member

    Joined:
    Nov 25, 2012
    Messages:
    8,233
    Location:
    Visalia, CA
    Even on straight (not curvy) intersections:

    When I bought AP2 in March 2017, it was really bad at any breaks in lane markings such as expected at most intersections.

    Now, 1.5 years later, it's much better but it is still unable to navigate in certain wide intersections that don't have lane markings.
     
  8. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    416
    Location:
    San Francisco, CA
    I'm also inclined to the view that the path prediction shown on the video is being generated entirely from video input. I don't actually know, personally, if there's any evidence of maps being used - I defer to others on that. I just thought the question was worth posing since the path prediction seems to be very good, and I wanted to know if I could be confident that it definitely wasn't looking ahead by using some sort of maps input.

    That the question seems relevant is, to me, a symptom that the feature is impressive.
     
    • Like x 2
    • Informative x 1
  9. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    416
    Location:
    San Francisco, CA
    I should add that, in my review of the binaries I was unable to find any evidence of map input being used and I consider that to be pretty strong evidence that it is not. But of course, if @verygreen has seen map information flowing into AP2 then that should also be weighed.
     
  10. pyraca

    pyraca Member

    Joined:
    Mar 3, 2018
    Messages:
    24
    Location:
    San Jose, CA
    Observations driving on AP and whether it uses Maps or not.

    1. When in exit only lanes from HW (like US 101N at Marine Parkway) the car will slow down and correctly take the exit on its own. It will stop if there are cars at the traffic light at the end of that exit. Once light turns green, cars ahead move and turn right, I am the only one heading forward, it does make this split second decision to go forward & then it drives through what appears to be pretty unmarked intersection (albeit at 25Mph). it somehow must know that the exit lane is being taken, and that most likely comes from maps, no ?

    2. When I took a trip up north through and was at Shasta, CA the driving in the hills was interesting on AP since the car did in some cases slow down when taking curves in these curvy parts. It was a deliberate, ahead of the turn slow down by 5-10mph from the top AP speed that I had set. I would have blown by without speed reduction ... again it could have been visual only but that behavior I thought changed at some point through one of the releases after the butter smooth AP from march this year.

    3. Counter argument is that on some exits, especially tight right loop interchanges (South bound 101, to 237 at Mt View, CA) the car will slow down but does not always make the loop in a way that I find comforting. A good driver will adjust the steering wheel in such cases minimally and in some will find the "right" position to take that loop - the car kinda keeps going forward and guessing almost.

    4. Highway interchanges (long right or left turns between highways) are taken also with reduced speed but not to the extend that exits are.
     
    • Informative x 1
  11. arg

    arg Supporting Member

    Joined:
    Aug 22, 2012
    Messages:
    1,791
    Location:
    Cambridge, UK
    Release notes apparently claim a feature to route using HOV lanes or not according to a config setting.

    How can that work without HD maps?
     
  12. mongo

    mongo Well-Known Member

    Joined:
    May 3, 2017
    Messages:
    12,857
    Location:
    Michigan
    Low def geofencing, like speed limits?
     
  13. mitchellh3

    mitchellh3 Member

    Joined:
    Nov 18, 2016
    Messages:
    162
    Location:
    Los Angeles, CA
    It hasn't been confirmed yet, but the expectation is that V9 is probably using some form of "HD" maps (how HD is tbd) specifically for highways (Navigation on Autopilot).

    @jimmy_d above was only analyzing and discussion V8 NNs. I don't think anyone has gotten a hold of the new NNs yet. And no one has published a public breakdown of the V9 firmware.
     
  14. Cosmacelf

    Cosmacelf Well-Known Member

    Joined:
    Mar 6, 2013
    Messages:
    8,229
    Location:
    San Diego
    HD maps can mean different things to different development groups.

    It is often meant to mean localization. Ie. GPS tells you where you are within a few feet (if it is working correctly, sometimes it isn’t, in which case it’ll be off by 30 feet or so), but to get accuracy within a few centimeters, you need something else. The something else that Mobileye uses is a “map” of side of road signs. The speed limit signs, turn ahead signs, mile markers, points of interest signs, etc. Any sign that can be uniquely identified within a, say, 50 foot radius. The vision systems looks for and tracks these signs and does a continuous range estimation to them via visual cues (how big they are getting relative to the speed the car is driving). This is then compared to its HD map for the area and via dead recogning, can tell the AP software where it is within an inch or so.

    I’ve often thought this is an important missing ingredient from Tesla’s AP. Among other things, it is the difference between driving on a road you’ve never driven before and one you’ve driven a dozen times before. Humans drive new roads much more cautiously than roads they know well. It is asking a lot for a vision system to drive an unknown road as well as an experienced human when the vision system doesn’t know what the road should do in 100 feet.
     
    • Like x 2
  15. croman

    croman Active Member

    Joined:
    Nov 21, 2016
    Messages:
    4,624
    Location:
    Chicago, IL
    What other purpose is Tesla marking poles and other landmarks other than to create HD maps? If they aren't already created, they are being built.
     
    • Helpful x 1
  16. Snuffysasa

    Snuffysasa Member

    Joined:
    Feb 24, 2017
    Messages:
    487
    Location:
    Minneapolis

    Exactly, they are creating / using maps, or at-least getting ready to start creating/using HD maps
     
  17. lightningrob

    lightningrob Member

    Joined:
    Jun 8, 2014
    Messages:
    17
    Location:
    Sunnyvale, CA
    Thanks for the detailed analysis. I'm curious, how did you obtain the binaries and extract the code? What language or NN framework does the AP use? I'd love to know the mechanics of how you reverse engineered it if you don't mind sharing.
     
  18. Pale_Rider

    Pale_Rider Member

    Joined:
    Jul 28, 2016
    Messages:
    673
    Location:
    Houston, TX
    Pretty sure that is not something that needs to be shared here since Tesla likes to keep an eye on things here sometimes...
     
  19. CK_Stuggi

    CK_Stuggi Member

    Joined:
    Mar 14, 2015
    Messages:
    281
    Location:
    Stuttgart
    In regards to the landmarks I found this:

    TL;DR precise landmarks seem to enable localization down to 2-3cm.
     
  20. malcolm

    malcolm Active Member

    Joined:
    Nov 12, 2006
    Messages:
    3,072
    Overkill?

    Source: Superaccurate GPS Chips Coming to Smartphones in 2018
     
    • Like x 1

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC