TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

Seeing the world the wrong way in v9

Discussion in 'Autonomous Vehicles' started by verygreen, Nov 2, 2018.

  1. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    So I was wondering how does AP v9 works in right hand drive countries and opportunity presented itself so I jumped on it. Thanks to everybody involved in making this happen!

    Here's some footage from 18.40 in Japan.

    Sadly there was a bit of a setback where when too many objects appear in the frame the buffer we use to collect it in overflows and it shows as nothing is being detected for some time. You'll see this at the start of the second video - don't worry the AP sees everything still, just artifact of our data collecting that I noticed too late to do anything about.

    Highway->surface streets transition:


    Busy surface streets->highway. It's interesting that the toll gate is not perceived as an obstacle even when closed.


    Airport parking garage (really drives autopilot nuts), parking guard encounter also thee's a superlow flying plane at ~1:05 that AP does not see (I sooo wanted it to see and classify it! ;) ):


    While the core NN seems to be the same, there are some processing differences, in particular non-adjascent lanes are detected a bit differently so needed to update the display logic to show them correctly.
     
    • Informative x 7
    • Like x 6
    • Love x 5
  2. pkodali

    pkodali Banned

    Joined:
    Oct 22, 2017
    Messages:
    817
    Location:
    Sacramento
    Is there any evidence that autopilot is detecting obstacles at the moment? I thought there wasn't
     
  3. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    the driving space thingie is the closest to that.
     
    • Informative x 3
  4. Bladerskb

    Bladerskb Like how many times do i have to be right?

    Joined:
    Oct 24, 2016
    Messages:
    1,254
    Location:
    Michigan
    One thing i like about @verygreen is that he tells it as it is.
    He doesn't hype stuff up, or sugar coat and gloss over any negative and water (*cough kool-aid*) board himself with the positives.
    While we may not always agree and come to the same conclusion. He's an honest stand up guy doing the lawd's work.

    With that said. Tesla currently lacks detection for regular obstacle objects, debris, traffic light, traffic sign, overhead road signs, road markings, barriers/guardrails, curbs, cone (as far as i can tell), etc.
     
  5. jimmy_d

    jimmy_d Deep Learning Dork

    Joined:
    Jan 26, 2016
    Messages:
    401
    Location:
    San Francisco, CA
    Nice work!
     
    • Like x 1
  6. croman

    croman Active Member

    Joined:
    Nov 21, 2016
    Messages:
    3,375
    Location:
    Chicago, IL
    Tesla's been doing cones of all types I've encountered for some time. The rest is perhaps missing but obtainable. Thanks @verygreen so much for revealing more layers of autopilot.
     
    • Like x 1
  7. strangecosmos

    strangecosmos Non-Member

    Joined:
    May 10, 2017
    Messages:
    858
    Location:
    The Prime Material Plane
    @verygreen Do you know how often snapshots occur? For example, could you count how many snapshots you've had in your (cars) and divide the miles you've driven by that number? I'm wondering about the miles of driving per image captured. Is it 100, 1000, 10,000?
     
  8. Anner J. Bonilla

    Joined:
    Oct 10, 2014
    Messages:
    104
    Location:
    Miami FL
    If by snapshot you mean data captures for Tesla I noticed that they are not a fix distance and they seem to correlate to amount of issues you have in a drive.
    If you never cancel a lane change, use autopilot features it seems to not upload more than 10 MB on a 41 mile drive. I am working on getting more concrete numbers in the next week's. Anyways i use it on my 44miles of daily driving and every day it uploads about 200mb
     
    • Informative x 2
    • Love x 1
  9. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    the mandatory copy of calibrations occur about every 5 minutes, but these are just that.- calibrations.
    The autopilot trip logs are every time you hit park after driving for a bit. Also when something crashes or stops working there's a logs snapshot, I get some of those, e.g. every time the backup camera thread stops (almost once per drive).
    Also if you disengage AP, there's a very small snapshot with just coordinates of where it happened generated (no pictures).

    That's about it. My car does not get any trigger requests from tesla so number of images per mile driven is zero for me. Of the old "campaigns" I saw (that they seem to be doing from once per day to once per week) it could be up to a a dosen or two images/snapshots requested per campaign in total, sometimes more sometimes less and the triggers are typically very precise so you might not trigger any pictures for a particular snapshot easily (e.g. I saw an "accelerate towards an obstacle" trigger that I have never seen triggered). Common trigger I saw was towards overriding autopilot (e.g. accelerate when it wants to brake, actively steering and so on), those get like 1-2 per campaign.
     
    • Informative x 3
    • Love x 1
  10. strangecosmos

    strangecosmos Non-Member

    Joined:
    May 10, 2017
    Messages:
    858
    Location:
    The Prime Material Plane
    #10 strangecosmos, Nov 4, 2018
    Last edited: Nov 4, 2018
    Because it’s hacked?

    Woah.

    Fascinating, thank you. How often were these campaigns?

    Your car uploads 200 mb per day? :eek:

    The reason I ask is I was trying to work out how many still images Tesla might have in its training library. An average of 1 image per car per month / per 1,000 miles would mean around 2.5 million images, and an average of 1 image per car per week / per 250 miles would mean around 10 million images.

    10 million images * 10 minutes to label each image = ~1.7 million labour hours
    ~1.7 million labour hours * $25/hour = ~$43 million

    So labour cost would be no problem. I was then trying to see what the cost of GPU hours would be to do Efficient Neural Architecture Search (ENAS) to optimize against a library of 10 million 1280x964 images. It took 16 GPU hours to do ENAS on the 50,000 training images in the CIFAR-10 dataset, which are 32x32. I converted CIFAR-10 and the hypothetical Tesla dataset into pixels. ENAS optimized against 3.2 million pixels per GPU hour, and I just assumed the same would be true of the hypothetical Tesla dataset. I then looked at AWS pricing, which is $3.06 per GPU hour, and found that it would cost around $12 million for the 386 million GPU hours needed.

    The high-level theoretical question is whether the ceiling/bottleneck is data or compute. Does data run out before compute gets too expensive, or does compute get too expensive before data runs out? Assuming that Waymo has unlimited free compute, and knowing that Tesla has a limited R&D budget, does Tesla have an advantage in AutoML because of its larger fleet of cars?
     
    • Informative x 1
  11. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    I guess? Who knows.

    They are typically out every day to two days and they last a week (but could be preempted sooner). not every car gets every campaign.

    That's kind of easy to reach. one single 1fps snapshot from all cams is ~100MB.
     
    • Informative x 2
  12. strangecosmos

    strangecosmos Non-Member

    Joined:
    May 10, 2017
    Messages:
    858
    Location:
    The Prime Material Plane
    If every car is like Anner’s, that’s 150 million images uploaded per month!! :eek:
     
  13. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    nah, it's definitely not like that for every car. And they do come in bursts too I am sure.
     
    • Informative x 2
  14. Vitold

    Vitold Active Member

    Joined:
    Aug 10, 2015
    Messages:
    1,551
    Location:
    NM
    One thing I don’t like about his videos is that there is no disclaimer that all his videos are only the approximation of how AP is processing the information.
     
    • Disagree x 2
  15. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    huh? what do you mean by "approximation"? the objects and stuff are directly from the autopilot state, it's not like I have a parallel computing engine to do those or some such.
     
    • Like x 1
    • Funny x 1
  16. EvanLin

    EvanLin Member

    Joined:
    Oct 9, 2016
    Messages:
    65
    Location:
    Asia
    I am more curious about how AP handles Asian local roads.
    Since Autosteer is "banned" on local roads in Taiwan. (GPS locked)

    Thanks for this video to know AP isn't banned on local roads in Japan.
     
    • Informative x 1
    • Like x 1
  17. hiroshiy

    hiroshiy Supporting Member

    Joined:
    Apr 6, 2013
    Messages:
    2,276
    Location:
    Tokyo, Japan
    Cool!!
     
  18. Tezlanian

    Tezlanian Member

    Joined:
    Apr 24, 2018
    Messages:
    214
    Location:
    North America
    @verygreen Could you do a set in a more rural environment where "road" becomes a rather loose term sometimes to see how well the current version copes even if it cannot be enabled?
     
    • Like x 1
  19. strangecosmos

    strangecosmos Non-Member

    Joined:
    May 10, 2017
    Messages:
    858
    Location:
    The Prime Material Plane
    Anyway we could figure out how much data is being uploaded from the fleet? A ballpark figure?
     
  20. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,031
    Location:
    TN
    Ask people with wifi monitoring to contribute their upload data?
     
    • Like x 1

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC