TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

AP2.0 Cameras: Capabilities and Limitations?

Discussion in 'Autonomous Vehicles' started by lunitiks, Feb 20, 2017.

  1. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    Well, the fisheye and the B pillar cameras too...
     
    • Informative x 1
  2. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    3,406
    Location:
    Bay Area
    Yes, those too! I just meant the narrow and main cameras seem terrible for drawing nearby cars. I mean sure, if you only had EyeQ3's main camera, you can make do with it. But if you have 5 cameras looking in that general direction, the main camera might not be the optimal one.
     
    • Like x 1
  3. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    #183 AnxietyRanger, May 10, 2017
    Last edited: May 10, 2017
    Here is the traffic light shot (AP2.0 Cameras: Capabilities and Limitations?) composed of @verygreen 's tools' bw and red raw images through ImageJ and then Photoshop warping for rough perspective correction. Glare shield (lower half of image) cropped out, obviously.

    Note the red car on the right. :)

    With color adjustments:

    tcd3_traffic_lights_rgb_with_red_channel_edited.jpg

    With only warping and red channel + green and blue channels in bw, no color editing:

    tcd3_traffic_lights_rgb_with_red_channel_warped.jpg
     
  4. Bladerskb

    Bladerskb Like how many times do i have to be right?

    Joined:
    Oct 24, 2016
    Messages:
    1,326
    Location:
    Michigan
    #184 Bladerskb, May 11, 2017
    Last edited: May 11, 2017
    mobileye presentation from 2015. They list the fov of their eyeq3 camera.

    here's another video

     
  5. J1mbo

    J1mbo Active Member

    Joined:
    Aug 20, 2013
    Messages:
    1,324
    Location:
    UK
    Any chance you can add the B pillar images to this? Curious to see how wide the FSD view is, and how much overlap there is with the fisheye.
     
  6. bjornb

    bjornb Member

    Joined:
    Feb 12, 2015
    Messages:
    181
    Location:
    Norway
    Yes, I have extracted those as well, I'll add them later.
     
    • Helpful x 2
    • Like x 1
  7. lunitiks

    lunitiks (ง ͠° ͟ل͜ ͡°)ง

    Joined:
    Nov 19, 2016
    Messages:
    2,543
    Location:
    Prawn Island, VC
  8. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,051
    Location:
    TN
  9. bjornb

    bjornb Member

    Joined:
    Feb 12, 2015
    Messages:
    181
    Location:
    Norway
    I've added the B-pillar images as well, as you can see there are overlap between the B pillar camera and fisheye camera.

    forward_merged_incl_pillar.JPG

    forward_merged_incl_pillar.JPG
     
    • Informative x 7
    • Like x 4
    • Helpful x 1
  10. J1mbo

    J1mbo Active Member

    Joined:
    Aug 20, 2013
    Messages:
    1,324
    Location:
    UK
    That's pretty cool, thanks. Interesting as to how the contrast on the B pillar pics in the first collage is so different to the other cams.

    Random guess that at this junction, without the MINI (?) alongside, the left B cam would add 2-3 seconds of visibillity of 50mph traffic coming from the left, over what the fisheye alone can see. With the MINI there, the view is quite restricted, maybe giving a 1s advantage. (However, I imagine it would give a very clear view of pedestrians crossing the junction if there were any).

    Interestingly, the view from the right B cam looks to be out of sync, maybe 1s earlier than the fisheye? You can just see the nose of a dark car entering the frame. The fisheye shows it parallel to the bush.
     
    • Helpful x 1
  11. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    Great work @bjornb and I noted the same as @J1mbo.

    I earlier said the overlap between fisheye and B pillar cameras is maybe one fifth of the fisheye in from left. 15-20% overlap maybe on each side? Maybe someone can measure...
     
  12. bjornb

    bjornb Member

    Joined:
    Feb 12, 2015
    Messages:
    181
    Location:
    Norway
    I didnt spend any time adjusting the B pillar images so it may not be representative of the actual quality, but for some reason the main and narrow cams (the only cameras in use at the moment) does look better out of the box with just an 'equalize' adjustment in photoshop.

    From the drivingabout folder supplied by @verygreen I have looked for some relevant highway images with multiple lanes and cars close by and I notice that the camera captures are out of sync with each other. I guess they are not captured simultaneously.
     
  13. bjornb

    bjornb Member

    Joined:
    Feb 12, 2015
    Messages:
    181
    Location:
    Norway
    Another one:
    merged.JPG
     
    • Love x 3
    • Informative x 2
    • Like x 1
  14. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    Excellent @bjornb, very informative!
     
  15. lunitiks

    lunitiks (ง ͠° ͟ل͜ ͡°)ง

    Joined:
    Nov 19, 2016
    Messages:
    2,543
    Location:
    Prawn Island, VC
    #195 lunitiks, May 12, 2017
    Last edited: May 12, 2017
    (Warning: Rant.)

    Looking at those photos, I can't fathom why the Main and Narrow - and not the Fisheye - are in use in EAP.

    Why capture GBs of video, occupy hugely valuable processing power, blind the car from an excellent FOV, by activating these two cams in EAP? I don't get it.

    Redundancy? Well, to me it seems very unlikely that one of the three fwd cams (both fwd looking, millimetes apart, both on the same ECU), would fail / be blocked. I don't buy it.

    As of now, I clinge to the hope that EAP in actuality will make use of *all* cameras when the time is right. Just SW limited to certain features (autosteer+, smart summon etc).

    Thanks :-/
     
    • Like x 1
  16. chillaban

    chillaban Active Member

    Joined:
    May 5, 2016
    Messages:
    3,406
    Location:
    Bay Area
    I have a feeling they will end up using more cameras for EAP. It simply doesn't make sense to implement the promised EAP features using more restricted cameras when they have the hardware on board to make problems easier to solve.
     
    • Like x 2
  17. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    9,454
    The narrow cam gives roughly 1.6x the range of the main cam (and radar which is slightly more than main cam at 160m vs 150m).

    Also, reading EAP's description it's still very focused on highway use, where the longer range is more useful than the wider view (which is more for local driving).
     
  18. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,051
    Location:
    TN
    There is 1 second delay between captures, hence the discrepancies.

    The camera init is done by the AP software and it's possible that only those two get any serious adjustments on the sensor.
    Gotta read those dumped registers from the raw frames to see what the difference is, I guess.
     
  19. Bladerskb

    Bladerskb Like how many times do i have to be right?

    Joined:
    Oct 24, 2016
    Messages:
    1,326
    Location:
    Michigan
    #199 Bladerskb, May 13, 2017
    Last edited: May 13, 2017
    This is coming from someone who was against narrow camera being apart of EAP and being the "redundant forward looking cameras" that Elon talked about. He argued heavily against it. Now he acts like he's in the know. lol
     
  20. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    #200 AnxietyRanger, May 13, 2017
    Last edited: May 13, 2017
    While anything is possible, I'm not so sure about this. The 4 cameras vs. 8 cameras is a great options selling point. EAP is basically AP1 goal completed eventually, so ramp to ramp highway driving with better blind-spot detection using the backwards looking side cameras. I don't see where they would need additional cameras for that scenario - whereas I see clear benefits for them in making additional features available soon for cars with FSD...

    Making EAP better than described is not in their interests IMO. I guess one could make the argument they'll need more cameras for some auto lane change scenarios, but they'll still have the blanket of ultrasounds there as well... What I could see Tesla do is add some kind of 8 camera AEB to the entire fleet. That I could see them do... That would be a way to not tie it into EAP.

    Also, I can definitely see Tesla retiring EAP at some point and changing the packs. However, precedent is, current AP1 on AP2 HW, EAP, FSD separation will still be maintained for existing cars and you can/will have to pay to upgrade features...
     

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC