TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker or making a Paypal contribution here: paypal.me/SupportTMC

Tesla Model 3 will have the new Autopilot 2.0 with dual cameras

Discussion in 'Model 3: Ordering, Production, Delivery' started by Lucy Lee, Jul 18, 2016.

  1. Lucy Lee

    Lucy Lee New Member

    Joined:
    May 4, 2016
    Messages:
    3
    Location:
    San Francisco, CA
    Tesla model X that were recently delivered already has the Dual Facing Camera built in.

    Elon will announce a newer / better Autopilot 2.0 very soon and our Tesla Model 3s will have this built in!

    [​IMG]
     
    • Informative x 1
    • Dislike x 1
  2. SΞXY P100D

    SΞXY P100D Member

    Joined:
    Nov 6, 2015
    Messages:
    285
    Location:
    Jersey
    #2 SΞXY P100D, Jul 18, 2016
    Last edited: Jul 18, 2016
    Much sooner than expected! I thought it would take another year. Any pics of the dual camera system, and will it include enhancements recently tweeted by Elon Musk in response to the fatal AP crash?
     
  3. S'toon

    S'toon Knows where his towel is

    Joined:
    Apr 23, 2015
    Messages:
    1,971
    Location:
    SK
  4. jaguar36

    jaguar36 Member

    Joined:
    Apr 10, 2014
    Messages:
    859
    Location:
    NJ
    The model X has always had this housing. It doesn't have dual cameras, just the housing. Obviously AP 2.0 is coming soon, but there is no evidence as to if thats tomorrow, next month or a year from now.

    While the Model 3 will obviously have some sort of AP there is no evidence at all that it will be the same as the S/X current hardware, the 2.0 hardware or something else entirely.
     
    • Like x 2
  5. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    I wonder how Tesla was getting stereoscopic camera view for distance prior to dual cameras? Subaru's Eyesight is dual cameras and that is how it tracks vehicles front for distance and speed with just one camera?
     
  6. Snow Drift

    Snow Drift Member

    Joined:
    Feb 10, 2016
    Messages:
    205
    Location:
    NYC
    Subaru makes the best system thanks to dual cameras to see depth. They were as of 2014 the only OEM to pass all of the IIHS crash avoidance tests.

    MBLY believes they have a superior tech that can work off one camera. Clearly it is not better.
     
    • Dislike x 1
  7. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    Hopefully the T3 will have the stereoscopic cameras. Love the Eyesight system on the Subaru. Very smart system. An example if someone cuts in front of you on the freeway, the system knows how close they are and if they accelerating away from you or slowing in front of you and reacts accordingly. The blind side indicators likewise use the approaching car's speed to warn you. It's lane keeping requires active driver input. It activates if you start to drift out of your lane so you can't just turn it on watch Harry Potter movies, you have to drive the car which will help you not make mistakes.
     
  8. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    7,039
    #8 stopcrazypp, Jul 31, 2016
    Last edited: Jul 31, 2016
    Tesla (or rather the Mobileye chip) does so with visual processing. You can get a visual estimate of distance simply from knowing the height of the camera and the position of the bottom of the vehicle relative to the video frame. This is further confirmed by the radar data. Details here:
    http://www.mobileye.com/wp-content/uploads/2011/09/VisiobBasedACC.pdf

    Here's a link that talks about the Mobileye system in general:
    Exclusive: The Tesla AutoPilot - An In-Depth Look At The Technology Behind the Engineering Marvel

    Other's have also speculated depth from motion is used (where different frames are compared and you can get an estimate of depth by using a stationary visual reference, camera fps, and your vehicle speed; similar to how people do the 2 or 3 second rule for following distance).

    You do not need a dual camera to see depth, especially at the distances for vehicles. There are far more monocular visual depth cues than there are binocular ones.
    Depth perception - Wikipedia, the free encyclopedia

    Having dual cameras gives you more accuracy than is possible with a single camera, but it isn't strictly required.
     
    • Helpful x 3
    • Like x 1
  9. Cosmacelf

    Cosmacelf Active Member

    Joined:
    Mar 6, 2013
    Messages:
    3,399
    Location:
    San Diego
    I'll second what stop crazy app (damn spell check) said. Dual cameras are NOT needed for depth perception. In particular, two cameras spaced a few inches apart can't discern depth stereoscopically past 20-30 feet anyways IIRC. Instead you use motion, scene understanding algorithms, and radar to come up with a fused understanding of where everything is. The second camera is much more likely to be used as a short range wide angle view camera.
     
  10. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    Subaru's top rated (iihs.org) system which uses stereoscopic cameras seems to be a proven better, simpler system. That Tesla is going to stereoscopic cameras would confirm it. As noted above, in practice, it is a very sophisticated and effective system. In practice the range seems to be close to 300 feet when it first detects objects and gives audible warning. It will then pull within range (four different following distance settings) and start to slow.
     
  11. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    7,039
    #11 stopcrazypp, Jul 31, 2016
    Last edited: Jul 31, 2016
    The Subaru got the top rating because it was the only vehicle that came to a complete stop in a 25mph test collision avoidance test (I should note IIHS did not test any Tesla vehicles yet, so we don't know how it compares in the same test). It is not necessarily because of the superiority of stereoscopic cameras (the logic and rear end collision risk that an automaker programmed into the system can have a lot to do with it).

    Also, I should dispel the misconception that the two cameras that Tesla might release has to do with a stereoscopic system.

    Just to make people aware. Subaru's Eyesight system uses two cameras mounted 14 inches apart and has no front radar backup to confirm distance to objects in the front (the latest vehicles have side and rear radar for other systems, but still no front radar). Stereo systems rely on something called disparity mapping to generate a 3D depth map. It can't function properly even if only one of the cameras are blocked or misaligned.
    Embedded Vision: Detect Pedestrians | EE Times

    See how far the cameras are mounted from each other in the Subaru system:
    [​IMG]
    Subaru Forester review: The best small SUV thanks to EyeSight | ExtremeTech

    The Model X housing however has a camera housing that allows a distance of only around 3 inches apart for two cameras (I did a rough estimate using the Model X photo on the Tesla website):
    [​IMG]

    This is presumably to support the Mobileye EyeQ4 system (or something similar given recent announcement of the end of the partnership) which despite supporting up to three cameras, does not use the cameras for stereoscopic depth sensing purposes. Such a short distance between the cameras would make stereoscopic based depth sensing poor esp. for the long range.

    Rather the multiple cameras is used to support different field of views, suitable for objects/pedestrians both in normal (50 degree as in the current single camera), close (pedestrians in the city), and far range (past the 100-200 meters of today's system). Here's an article that describes this and an illustration that shows how the three different cameras would be used:
    [​IMG]
    The Linley Group

    Even the oldest version of the single camera Mobileye system can do vehicle detection out to 70 meters (230 feet) and 100 meters (328 feet) for version 2 using a cheap VGA camera. If using a higher resolution camera, that extends to 200 meters (656 feet). Tesla currently uses version 3 (which I presume may have even longer range).
    Vehicle Detection - Mobileye
     
  12. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    In the Subaru's case it is necessarily due to the stereoscopic cameras since they provide the depth perception that activates the braking system to stop. It likely was not only the cost efficiency but the accuracy that appealed to Subaru.

    Nature uses the same system. Human eyes are typically just 2-3 inches apart and provide excellent stereoscopic views and depth perception.

    More than the average driver that the system is trying to emulate.

    Using the stereoscopic cameras eliminates the need for the forward radar system and would provide a cost savings and a system upgrade at the same time. Something that would be appealing to Tesla with the T3 being Tesla's "economy car".
     
  13. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    7,039
    #13 stopcrazypp, Jul 31, 2016
    Last edited: Jul 31, 2016
    This is completely unfounded speculation. An automatic emergency braking system involves a lot more than simply depth perception. The available processing power and algorithm (which effects processing time), allowed speed differential (maximum difference between vehicle speeds where system activates, this is 20mph for the Eyesight system, only 9mph for the Volvo system), minimum and maximum activation speeds, maximum brake force applied by the system, maximum speed difference the system allows after activation, just to name a few I thought of. All this will affect whether a system would be able to pass that test.

    That ignores the distances we are discussing. Horizontal disparity (which a stereo system relies on to do depth sensing) is directly related to the distance between two eyes (or two cameras in this case).
    Stereoscopic perception of real depths at large distances | JOV | ARVO Journals

    So if a 14 inch distance in the Subaru system yields 80 m (262 ft) of range, 3 inches will yield only about 17 m (56 ft) of range for effective depth perception. This is definitely not acceptable or useful in an automobile setting and far below a radar system (which is 100m or more), even though for our eyes it is plenty useful. After the recent accident, there were calls for Tesla to have sensors for even longer range.
    Subaru Gives EyeSight to the Distracted

    Again, I should remind you that human depth perception relies largely on monocular cues (AKA single camera analogy) and only on binocular cues (AKA stereo camera) for the shorter distances.

    The idea with a Autopilot 2.0 is to improve upon the existing one, not a step backwards. I would not expect Tesla to eliminate the front radar sensor. It is still required to see in lower visibility conditions (something radar can handle, but a camera system can't).
     
  14. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    No it's not since the stereoscopic camera is the only detection system the Subaru has for braking it is key to providing the depth perception necessary. We are talking about the sensor input data,not the data processing of it which is the same in the all vehicles with auto braking.

    No it doesn't ignore it. It does note that three inches is more that humans have for depth perception which is the distance between the cameras in the Tesla.

    So human depth perception is only 56 ft?
     
  15. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    7,039
    This is completely false. I already pointed out large differences between the Subaru and Volvo system completely independent of the detection system. Subaru set their parameter such that a maximum closing speed of 20mph will still have the system activate. Volvo set it at 9mph (which pretty much insures it can't past the 25mph test). There are also systems that don't allow the car to autobrake to a stop (so it will never get full points on the test).

    I'm talking about binocular depth perception (ones that relies on two eyes or two images). This is done by comparing differences between two images (a close object will have a larger difference, while a far object will have less difference).

    Monocular depth perception (which can be accomplished by one eye or picture using visual cues) can extend much further. For example, when you look at an airplane in the sky, you can tell how far away it is just from the relative size of it, even though the distance is already far beyond the point where comparing the two images from your different eyes can give you an idea.

    Also a camera and human eye has completely different resolutions. The typical VGA camera in many systems only has 0.3MP. The human eye is counted at minimum around 10MP near the center and ranges to 576 MP if you count peripheral vision and how the eyes can move (unlike a fixed camera). My calculation is only applicable to the assumption that a similar camera is used in both cases (do note, there is a limiter on adopting very high resolution sensors, given the processing power required goes up proportion to the resolution).

    Main point is, if Tesla intended their set up to be a stereoscopic camera, they would put them much further apart (Subaru deliberately chose that distance so that their system can match a bare minimum radar setup at 80m). You have also completely ignored my point that Mobileye already said the other two front cameras will not be used as a stereoscopic system, but rather to allow front cameras with different fields of view.
     
    • Like x 1
  16. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    Subaru braking will activate at 60 mph in dynamic cruise mode. It can bring the vehicle to a stop in 126 feet from 60 mph and will if you are cruising at 60mph and traffic dead stops 127 feet in front of you.

    Auto braking has different parameters when dynamic cruise is off and that is 30 mph max speed per the manual to bring vehicle to safe stop. The Insurance Institute for Highway Safety uses 25 mph for its test speed for auto-braking.

    Both, auto braking and dynamic cruise, use the stereoscopic camera as the sole information source.

    Subaru has no radar, just the stereo cameras. Tesla will be able to use 3 in. camera separation (same as human eye) just as well, just takes a faster computer since the information has less differential to use. Tesla may still use the radar sensor along with the new stereoscopic cameras since is uses radar for other functions like auto-park where the camera could not see things at bumper level up close. Subaru warns about objects the camera's can miss for cruise and auto braking.
     
  17. raysspl

    raysspl Member

    Joined:
    Mar 2, 2016
    Messages:
    132
    This is very very good to hear! Can't wait to see real world results on this upgrade.
     
  18. stopcrazypp

    stopcrazypp Well-Known Member

    Joined:
    Dec 8, 2007
    Messages:
    7,039
    #18 stopcrazypp, Aug 2, 2016
    Last edited: Aug 2, 2016
    The first version had a 19mph closing speed, but 2nd version updated to 30mph. The system can still react at a higher speed, but might not stop in time.
    http://media.subaru.com/pressrelease/562/123/subaru-debuts-next-generation-eyesight-system

    But the specifics of the exact mph doesn't matter. My main point is that the fact that the Volvo system had a different and lower 9 mph setting that made sure it won't get the top score and that such a speed differential setting would make a huge difference in the test results regardless of the detection system being used. Basically your point ignores there are many independent variables to a vehicle getting a top score and the detection system is only one of them.
    Subaru Gives EyeSight to the Distracted

    Actually more processing power would not solve the problem. The computer can't process something the sensors didn't detect. Tesla would need to use two higher resolution sensors that is 4.7x the horizontal resolution of Subaru's (given sensors typically increase in both horizontal and vertical resolution at the same time, in practice, that might mean a sensor with 22x the MP). That's going to be very expensive (plus it also ignores optics: sensor size and lens size matters also, so you can't just throw a high resolution sensor in). It is far cheaper to just make the separation distance further (or accept that the system will have a poor depth perception range).

    But the known evidence points against this speculation regardless. I didn't look very carefully the first time around, but digging around, I looked more closely at the block diagram that was recently leaked about the updated autopilot hardware and guess what it says:
    Triple Cam - Main
    Triple Cam - Narrow/Mono
    Triple Cam - Fisheye
    [​IMG]
    That matches exactly the same illustration I posted before about the EyeQ4 system:
    [​IMG]
     
    • Informative x 1
  19. palmer_md

    palmer_md Member

    Joined:
    Sep 26, 2012
    Messages:
    718
    Location:
    Sacramento, CA


    Pretty cool video explaining some of the 3d effects with technology.
     
  20. EaglesPDX

    EaglesPDX Member

    Joined:
    Jun 8, 2016
    Messages:
    405
    Location:
    PDX
    @stopcrazypp "Actually more processing power would not solve the problem. "

    The closer the two cameras are together the less diiferentiation so the computer would need to be able to make up the loss.

    @stopcrazypp "The computer can't process something the sensors didn't detect. Tesla would need to use two higher resolution sensors that is 4.7x the horizontal resolution of Subaru's (given sensors typically increase in both horizontal and vertical resolution at the same time, in practice, that might mean a sensor with 22x the MP). That's going to be very expensive."

    Well...we wouldn't want a Tesla to get as expensivel as a Subaru.
     
    • Funny x 1

Share This Page