Separate names with a comma.
Discussion in 'Autopilot & Autonomous/FSD' started by lunitiks, Feb 20, 2017.
Well, the fisheye and the B pillar cameras too...
Yes, those too! I just meant the narrow and main cameras seem terrible for drawing nearby cars. I mean sure, if you only had EyeQ3's main camera, you can make do with it. But if you have 5 cameras looking in that general direction, the main camera might not be the optimal one.
Here is the traffic light shot (AP2.0 Cameras: Capabilities and Limitations?) composed of @verygreen 's tools' bw and red raw images through ImageJ and then Photoshop warping for rough perspective correction. Glare shield (lower half of image) cropped out, obviously.
Note the red car on the right.
With color adjustments:
With only warping and red channel + green and blue channels in bw, no color editing:
mobileye presentation from 2015. They list the fov of their eyeq3 camera.
here's another video
Any chance you can add the B pillar images to this? Curious to see how wide the FSD view is, and how much overlap there is with the fisheye.
Yes, I have extracted those as well, I'll add them later.
DRIVE PX 2
What I wish for is the sdk from here: https://developer.nvidia.com/driveworks-sdk-downloads
Anybody happen to have a copy they can share?
I've added the B-pillar images as well, as you can see there are overlap between the B pillar camera and fisheye camera.
That's pretty cool, thanks. Interesting as to how the contrast on the B pillar pics in the first collage is so different to the other cams.
Random guess that at this junction, without the MINI (?) alongside, the left B cam would add 2-3 seconds of visibillity of 50mph traffic coming from the left, over what the fisheye alone can see. With the MINI there, the view is quite restricted, maybe giving a 1s advantage. (However, I imagine it would give a very clear view of pedestrians crossing the junction if there were any).
Interestingly, the view from the right B cam looks to be out of sync, maybe 1s earlier than the fisheye? You can just see the nose of a dark car entering the frame. The fisheye shows it parallel to the bush.
Great work @bjornb and I noted the same as @J1mbo.
I earlier said the overlap between fisheye and B pillar cameras is maybe one fifth of the fisheye in from left. 15-20% overlap maybe on each side? Maybe someone can measure...
I didnt spend any time adjusting the B pillar images so it may not be representative of the actual quality, but for some reason the main and narrow cams (the only cameras in use at the moment) does look better out of the box with just an 'equalize' adjustment in photoshop.
From the drivingabout folder supplied by @verygreen I have looked for some relevant highway images with multiple lanes and cars close by and I notice that the camera captures are out of sync with each other. I guess they are not captured simultaneously.
Excellent @bjornb, very informative!
Looking at those photos, I can't fathom why the Main and Narrow - and not the Fisheye - are in use in EAP.
Why capture GBs of video, occupy hugely valuable processing power, blind the car from an excellent FOV, by activating these two cams in EAP? I don't get it.
Redundancy? Well, to me it seems very unlikely that one of the three fwd cams (both fwd looking, millimetes apart, both on the same ECU), would fail / be blocked. I don't buy it.
As of now, I clinge to the hope that EAP in actuality will make use of *all* cameras when the time is right. Just SW limited to certain features (autosteer+, smart summon etc).
I have a feeling they will end up using more cameras for EAP. It simply doesn't make sense to implement the promised EAP features using more restricted cameras when they have the hardware on board to make problems easier to solve.
The narrow cam gives roughly 1.6x the range of the main cam (and radar which is slightly more than main cam at 160m vs 150m).
Also, reading EAP's description it's still very focused on highway use, where the longer range is more useful than the wider view (which is more for local driving).
There is 1 second delay between captures, hence the discrepancies.
The camera init is done by the AP software and it's possible that only those two get any serious adjustments on the sensor.
Gotta read those dumped registers from the raw frames to see what the difference is, I guess.
This is coming from someone who was against narrow camera being apart of EAP and being the "redundant forward looking cameras" that Elon talked about. He argued heavily against it. Now he acts like he's in the know. lol
While anything is possible, I'm not so sure about this. The 4 cameras vs. 8 cameras is a great options selling point. EAP is basically AP1 goal completed eventually, so ramp to ramp highway driving with better blind-spot detection using the backwards looking side cameras. I don't see where they would need additional cameras for that scenario - whereas I see clear benefits for them in making additional features available soon for cars with FSD...
Making EAP better than described is not in their interests IMO. I guess one could make the argument they'll need more cameras for some auto lane change scenarios, but they'll still have the blanket of ultrasounds there as well... What I could see Tesla do is add some kind of 8 camera AEB to the entire fleet. That I could see them do... That would be a way to not tie it into EAP.
Also, I can definitely see Tesla retiring EAP at some point and changing the packs. However, precedent is, current AP1 on AP2 HW, EAP, FSD separation will still be maintained for existing cars and you can/will have to pay to upgrade features...