Here is a quick demo of a 360 degree video I made using data from the autopilot cameras. Shoutout to @verygreen for providing me with the raw footage. Video was generated in after effects and current version in missing mian, narrow, and rear cams. (main camera is an after effects rendering issue, and I don't have footage from the narrow and rear cameras.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Autopilot in 360 Degrees
- Thread starter pkodali
- Start date
I meant as a 360 degree or VR video.
Like I can take a 360 camera and use that footage within VR so I can look around while the video plays.
Edit: Looking at it more I think what I requested is exactly what you posted.
This should play as a 360 degree video on chrome. I can pan around it
Here is an updated version with data from the main camera and a speed overlay
Last edited:
BOOM!
Showcases the blind spot i talked about when a car is directly behind you on a straight road. With the backup camera being so easily defeated.
I wish i could stop being right... lol
Showcases the blind spot i talked about when a car is directly behind you on a straight road. With the backup camera being so easily defeated.
I wish i could stop being right... lol
when cars are mostly aligned in their lane. you actually don't see the car behind you unless its a big truck or something.
We already know the back camera is problematic, with the additional issue that verygreen pointed out making it worse.
That means someone behind you approaching fast and then quickly changing lanes would be completely missed by the rearward cameras until it was too late. If you were attempting to change lanes at that time, it would be an automatic crash.
Funny enough, Tesla completely misrepresented the FOV of their rearward cameras. Interestingly, Mobileye put their rearview camera inside the car i guess to protect it from the elements.
Hardware capable of L5?
I don;t have data from the backup camera, but there shouldn't be a blind spot with the backup camera in there
I know but the backup camera is easily compromised in light rain and snow.
It's interesting to see how the repeater cams show the lane lines of your lane fairly well. It could be used to verify whether or not you're in your lane / centered as an extra layer of redundancy.
At least on the highway, looking around, it seems like it would provide enough information for a human to perform any driving maneuvers required. I'd love to see the same for cities so we can determine how well this would work at intersections, etc.
At least on the highway, looking around, it seems like it would provide enough information for a human to perform any driving maneuvers required. I'd love to see the same for cities so we can determine how well this would work at intersections, etc.
Been playing around with the warping, but in theory if verygreen can send some more footage, i should be able to graft it onto the current model
Once again you claim your right about a concern that most anyone that's owned a Model S/X/3 has expressed.
That the rear camera gets so dirty it's barely useable. In my experience it was atrocious with my Model S, and it hasn't been too bad with my Model 3. That's not saying much as rain season has just started so I'll see in a couple months how well it does. I'll definitely be cleaning it off more often now that it's being used more.
verygreen expressed a bigger concern in that there is a bug or electrical issue that sometimes causes the rear camera not to feed video.
This is the number one reason that I don't think even L3 driving is doable in a Tesla X/S/3 without something changing in the sensor suite.
Regardless of L2, or L3 the Tesla AP is going to be extremely finicky about wanting good visibility in all cameras, and will likely display an error message if it's not.
So I guess it's perfectly fine for people in California and less perfect for those of us outside of California
backup camera is very unstable. It does not work more often than it does. Apparently there was no pressure to fix it since it's not really visible by the end user (until v9 anyway, with v9 you too an see the camera does not work when the car directly behind you at a traffic light don't show up).
anyway I *finally* managed to get a long enough streak of backup cam footageto be useful, or so I think, anyway. Now needs some processing to make all the other cams show up.
I'm a little confused about the comment where it's not really visible to the user.
With V9 the entirety of TMC was woken up to the fact that a lot of S/X owners in fact do use the rear view camera while moving forwards. If it was glitchy they'd report it. Now I've seen it glitch while in reverse where it doesn't come up, but that's fairly rare.
Is that the glitch you're talking about or is there a glitch with the feed going to the AP2 computer?
A long time ago I designed a digital camera system using a TI OMAP processor where I ran into similar issue with the video feed either stopping or becoming corrupted. It only happened during an electrical fast transient burst (like during an ESD event). What was happening is the ISP (image signal processor) would glitch. It got confused when it didn't have the exact number of lines it expected because the transient bursts upset the sync signals. It was a fairly long cable run so that wasn't entirely unexpected, but I never could figure how to reset just the ISP in an elegant manner without having to reset the entire processor. So I solved it by using a CPLD to recreate the proper sync signals if it got messed up. Where line by line it made sure the timing was correct. Sure the video still got momentary corrupted during an event, but at least it went back to normal when the event was over.
One of these days I might try do a bunch of +15KV air discharge events near the rear view camera on my Model 3 to see if it causes it to glitch, and whether its momentary glitch or locked up until a reboot.
Well.
It used to be both. See Dangerous backup cam freeze?
So they fixed it just for the mcu display, but internally the feed still cuts out often and regular people ha no visibility into this whole thing. And now they do.
The logs indicate it's a loss of sync, it attempts 10 retries and then bails out until the next system restart.
Code:
09:18:34.00000-0700 ( 94.06176) W camera_stream.cpp:891]: Sleep for a while to wait for backup camera to get ready
09:18:36.00000-0700 ( 96.06176) W camera_stream.cpp:899]: Successfully starting capture on backup camera
09:18:36.00000-0700 ( 96.16276) W aptina_csi_controller.cpp:2661]: NvMediaIPPComponentGetOutput failed for position BACKUP; error = 3
09:18:36.00000-0700 ( 96.16276) W ti_deserializer.cpp:237]: TI Deserializer: Register 0x24 of deserializer 2 is 0x84
09:18:36.00000-0700 ( 96.16276) W ti_deserializer.cpp:260]: TI Deserializer: RX error on port 2 of deserializer 2
09:18:36.00000-0700 ( 96.16376) W ti_deserializer.cpp:297]: TI Deserializer: Port 2 of deserializer 2: Status Registers: 0x4D = 0x93 and 0x4E = 0x45; Parity Error: 0; Lock Error: 1; CRC Error = 0; Line Length Unstable = 0; Line Length Changed = 1
09:18:36.00000-0700 ( 96.16376) W camera_stream.cpp:951]: Failed to grab a frame for camera backup: Failed to get frame.
09:18:37.00000-0700 ( 96.26376) W aptina_csi_controller.cpp:2661]: NvMediaIPPComponentGetOutput failed for position BACKUP; error = 3I wonder what's so special about the backup camera. Why would it freeze, and not the others? Is it because the signal is split where it goes to the MCU and the drive computer?
