TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

Tesla autopilot HW3

Discussion in 'Autopilot & Autonomous/FSD' started by verygreen, Jan 4, 2019.

  1. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,425
    Location:
    TN
    Like I said they are still in the bringup phase for it which explains the extra debug.
     
    • Like x 1
  2. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,425
    Location:
    TN
    #62 verygreen, Jan 4, 2019
    Last edited: Jan 4, 2019
    yes, there are (of hw2.5) https://photos.app.goo.gl/7w6LQlQrxjhISILY2

    they have some deserializers that are connected to the cameras and then get that stuff via .... csi?
     
    • Helpful x 1
    • Like x 1
  3. tomc603

    tomc603 Member

    Joined:
    Jul 28, 2017
    Messages:
    731
    Location:
    NH, US
    Those look a lot like the photos featured on Electrek last year. Sadly, the clearest components are the GPS, memory, and GPUs. Either way, the A53/A72 based SoCs I've found all have CSI available natively, so that probably makes the most sense. CSI is also enabled in the kernel config file, which really wouldn't make much sense unless you had those capabilities and were using them.
     
    • Informative x 2
  4. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,425
    Location:
    TN
    yeah, I guess I can take more detailed ones but there's little point at this stage and there were some Chinese reports but the pictures were all downsampled, though all components were actually listed.

    Only tesla has hw3 samples at this stage I imagine, and I somehow doubt they are sharing ;)

    Anyway, feel free to ask your kernel questions here or in a PM, but keep in mind I only have hw3 code, but nowhere to run it. I obviously have hw2.5 hw so a lot more is possible there (but again very little is unknown about it at this time)
     
  5. tomc603

    tomc603 Member

    Joined:
    Jul 28, 2017
    Messages:
    731
    Location:
    NH, US
    This isn't a limitation of the radar sensors used, but rather in how radar works and must be implemented in a car. First, the car uses doppler radar, so it uses a difference of speed to "see" objects. Second, all still objects effectively "look" the same to the radar sensor. So, passing a road sign looks like a giant flat reflection, and the system would want to filter it out. Likewise, a stopped truck basically blends into the surroundings and gets filtered out.

    Something to note is that every vehicle using radar-based EAB has the exact same problem. If someone changes lanes quickly in front of you to avoid a stationary object, you're going to hit that object.

    Remember that as the photosites in a sensor get smaller and pixel density increases, electrical noise becomes a bigger and bigger problem. So doubling the resolution on a camera isn't necessarily going to give you double the usable data for the network. Further making things complex, a camera working at a high frame rate with a high resolution is going to produce an image that requires more pixels to be processed, which will require more RAM, faster busses, and bigger networks. Most existing neural nets I've seen do an unbelievable job at detecting objects and accurately classifying them on super low resolution images.

    At 100 MPH, a vehicle would travel 147 feet per second. With a camera operating at 60 fps, capturing two frames for motion estimation, the object in the camera would travel around 9.7 feet. If the object is 100-200 feet away, it's not too big a deal. At 300 feet per second, that object travels 20 feet in the same 2-frame span. There aren't too many situations in which two vehicle approach one another with a speed difference of 300 feet per second, so that would be a pretty rare case. But even still, with the object covering only 20 feet of distance, the system could have worked out what it is, where it is, where it's headed, and how fast it's getting there.

    At distances beyond 100-300 feet, you're probably wisest to rely on lidar or optical cameras with optics on them that are optimized for distance. This requires no increase of resolution, and would actually reduce the field of view anyway.
     
    • Like x 4
    • Informative x 1
  6. Vitold

    Vitold Active Member

    Joined:
    Aug 10, 2015
    Messages:
    1,692
    Location:
    NM

    I think reference to single exposure is the key and it means that ISP can combine two frames taken at different exposure, like HDR photos on the smartphone.
     
  7. MarkS22

    MarkS22 Member

    Joined:
    Apr 6, 2015
    Messages:
    796
    Location:
    Morris County, NJ

    While this is generally true (i.e. more pixels mean more noise, less light gathering), that ignores the fact sensor technology is improving dramatically. Advancements like back-illumination can offset some of these disadvantages.

    For now, the optical trifocal cluster seems to be compensating for resolution. But at a certain point, the quality and cost of vastly superior sensors will be available. I’m curious if the wiring is designed to handle this to make a hypothetical HW4.0 sensor suite possible if needed. We know they can swap out the processors easily. Will the dead end be sensors?
     
  8. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,425
    Location:
    TN
    But the camera sensors they use already have "HDR mode" where they give you 10 bits of data which is aplenty as we saw from the raw images even with a lot of contrast (like traffic light with sun directly behind it).

    Combining two pictures in a moving vehicle makes even less sense due to potentially huge movement between the frames.
     
    • Informative x 4
    • Like x 2
  9. MarkS22

    MarkS22 Member

    Joined:
    Apr 6, 2015
    Messages:
    796
    Location:
    Morris County, NJ
    Most likely for spatial reconstruction, no? Seems like a good way to use a 2D source to generate 3D data.
     
    • Like x 1
  10. verygreen

    verygreen Curious member

    Joined:
    Jan 16, 2017
    Messages:
    2,425
    Location:
    TN
    yes, but not for "HDR"-like dynamic range boost.
     
  11. MarkS22

    MarkS22 Member

    Joined:
    Apr 6, 2015
    Messages:
    796
    Location:
    Morris County, NJ
    Right. I meant it’s much more likely being used for 3D solving than HDR.

    They’d be better off putting an ND filter on one of the three forward cameras to add to dynamic range if it became a major issue.
     
  12. tomc603

    tomc603 Member

    Joined:
    Jul 28, 2017
    Messages:
    731
    Location:
    NH, US
    Or they could use the less expensive cameras to increase margins that much more. Again, given more data you will require more processing, and more of that processing will be done on useless data (areas of the image that don't contain important or meaningful objects). We already "know" Tesla is downsampling the images they gather today to limit processing requirements, so I don't see much use in going further for such a minimal gain of functionality.

    Anyway, the cabling for the cameras is likely just coaxial given what the connectors look like. They could be carrying LVDS, GMSL, or even Ethernet. We know nVidia uses GMSL on their boards, and we know Continental's components use Ethernet or LVDS. GMSL does 1.5 or 3Gbit/sec, Ethernet obviously can be 100M, 1G, 2.5G, 5G, 10G, or 25Gbit/sec, and LVDS is really less of a particular signalling rate rather than a PHY signalling method, but lots of LVDS devices communicate at 655M, 1G, and 3Gbit/sec. No matter what, swapping the camera and MCU at the same time could mean any future protocol and speed could be used as long as the cabling is shielded and can handle the signalling rate. Like moving from 10Mbit to 100Mbit to 1000Mbit to 10000Mbit Ethernet on the same Cat-6 cable.
     
    • Like x 1
  13. MarkS22

    MarkS22 Member

    Joined:
    Apr 6, 2015
    Messages:
    796
    Location:
    Morris County, NJ
    Resolution, sure. But you can still have better images without more data. Dynamic range, low-light sensitivity, and color come to mind. Just like the difference between AP2.0 and 2.5.

    While I understand your point of keeping it cheap and simple, there will certainly be a time (HW4?) where camera data will be the limiting factor of neural network recognizing objects faster and more accurately. And, by that time, I wouldn’t be surprised if better sensors cost less than the current ones.

    That said, coax should be pretty future-proof. I appreciate that insight. Are there photos of the connectors on the cameras anywhere?
     
    • Like x 1
  14. lunitiks

    lunitiks Cool James & Black Teacher

    Joined:
    Nov 19, 2016
    Messages:
    2,708
    Location:
    Prawn Island, VC
    They are coax cables yes. Rosenberger fakra connectors.

    APE2.0 unit:

    [​IMG]
     
    • Informative x 4
    • Helpful x 2
    • Like x 1
  15. lunitiks

    lunitiks Cool James & Black Teacher

    Joined:
    Nov 19, 2016
    Messages:
    2,708
    Location:
    Prawn Island, VC
    #75 lunitiks, Jan 4, 2019
    Last edited: Jan 5, 2019
    triplecam.jpg
    repeater.jpg
    pillar.jpg
    buc (1).jpg
    buc (2).jpg
     
    • Informative x 4
    • Like x 1
  16. lunitiks

    lunitiks Cool James & Black Teacher

    Joined:
    Nov 19, 2016
    Messages:
    2,708
    Location:
    Prawn Island, VC
    #76 lunitiks, Jan 5, 2019
    Last edited: Jan 5, 2019
    According to Tesla's wiring diagram, the b-pillar and side repeater cams (and gps!) are wired with D302. I believe this means it's a LEONI Dacar 302, 0.4 mm2 coax. You can tell Tesla uses Dacar 535 for the backup cam, ref. the picture above.

    The three forward cameras are wired with RG316 according to the schematics. Dunno what brand
     
    • Informative x 3
  17. electronblue

    electronblue Active Member

    Joined:
    Oct 1, 2018
    Messages:
    2,073
    Location:
    Earth
    #77 electronblue, Jan 5, 2019
    Last edited: Jan 5, 2019
    Okay so something I think others here missed. Did Exynos 7650 and 7800 with A72 core actually come out or were they just leaked or announced in 2015? When Tesla would have gotten prototypes. But did those come to the market? There is precious little on them online after the 2015 preliminary info. @verygreen

    Because all this from Tesla aka @shiny.sky.00 would fit with Tesla prototyping on 7650/7800 but then plans changing and them going somewhere else for the SoC. Which would indeed point to the here analyzed AP 2.5 firmware having leftovers from a prototype board.

    Then again all this would also with with @shiny.sky.00 just being a troll with poor googling skills. :D

    Anyway great job by @verygreen and co so thank you for that.
     
  18. jimmysanto

    jimmysanto New Member

    Joined:
    Jan 5, 2019
    Messages:
    2
    Location:
    San Francisco
    @verygreen the text on the attached images are too blurry. Is a higher resolution version available? Thanks!
     
    • Like x 2
  19. lklundin

    lklundin Active Member

    Joined:
    Oct 10, 2014
    Messages:
    2,088
    Location:
    Munich
    I gave you a disagree alone for posting to a thread without reading its opening post - if you just want to demonstrate your knowledge on this topic without discussing other posts, you are free to create your own thread.
     
    • Like x 7
  20. bjornb

    bjornb Member

    Joined:
    Feb 12, 2015
    Messages:
    181
    Location:
    Norway
    LVDS according to this:
    从TeslaAP2.0/2.5运算单元看未来无人驾驶域控制器的设计趋势 | 雷锋网
     
    • Informative x 2
    • Like x 2

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC