-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Neural Networks
- Thread starter lunitiks
- Start date
That is true and I am sceptical about his procedure myself. I'm just curious if there are any hints in the code/NNs/whatever that support this claim. Because...
Just for reference, someone else on TMC did a similar test some while ago, with a completely different result.
If we assume that both tests succeeded in blocking out the respective cameras we get to this conclusion:
AP2 before 2018.10.4:
Normal performance with fisheye and main camera.
Impaired performance with only fisheye.
AP2 after 2018.10.4:
Normal performance with only fisheye camera.
The results of the old test support the hypothesis that Tesla used two of the forward cameras in a stereo setup, since the performance drops heavily when one of the active cameras is covered.
The results of the current test support the hypothesis that Tesla either completely moved away from the stereo setup, or if not, that their stereo setup is either improbably robust or offers no benefit at all.
It would be very interesting if someone could try to repeat this test and confirm those findings.
The analysis done here based off the neural net files directly contradicts the theory that the fisheye is a primary driving camera. I'd highly recommend reading @jimmy_d's post analyzing the NN changes before and after 10.4. There's still a "main_narrow" net that obviously performs lane recognition duties and a separate fisheye net that seems to do a little more but is nowhere near as advanced as the main net.
With that said, I do agree that stereo vision is likely only useful in extremely limited situations, and just driving around a little bit is unlikely to uncover that, much like if you asked a human to drive around with one eye covered.
And that's exactly why I'm posting this here. It's an interesting finding that possibly contradicts our current understanding of their vision stack. I don't know how privy @jimmy_d is to AP's internal workings. I don't know if there is any explicit reference to which NN runs with which camera data. Just because Tesla might have used one NN for the main and narrow cams doesn't mean that they changed it now. This is nothing unheard of in software projects, that you change the functionality but the name stays the same because multiple other things are referencing it.
All of this is very murky and as far as I understand it, @jimmy_d only took a look at the definition files of said NNs and thus he can only really talk about the architecture of the NNs itself.
There are certainly other hints that they changed things around, from intangible things like the massively improved performance to the change of the internal name of the assumed main/narrow NN. They could be very well running this "main/narrow" NN on the fisheye now.
And please stop assuming that I haven't read the currently available information in this thread when I'm constantly referencing it. This seriously puts me off of posting here in the first place. All these technical threads here rely on a few select people that have inside information with hard facts being posted far and in between. In the end most of this stuff is speculation. So why can't we discuss all the possibilities, especially when there are experimental findings that can be verified by anyone with an applicable car.
Maybe instead of dismissing everything grab a roll of tape and do some tests yourself. I would love to do it but I don't have a Tesla.
Welcome to the forums @SpotfireY !! Let me assist a little bit otherwise, I'm not going to rehash all these discussions over and over again, it's pointless, either people believe or they don't believe and we'll that's everyones prerogative. There is not a whole lot of speculation going by some of us, the reason for the "awesome" improvement in the NN is because of the rewrite of the the NN and the much much larger dataset used to train said NN. Please search around the forums and looks for posts by @verygreen and @jimmy_d, there's A LOT more information around, it's not all in this thread, not even close.
Last edited:
I can't tell if you're being sarcastic or genuine... I've been reading these threads since AP2 first appeared. This is the biggest progress they have made in years. I'm genuinely curious if they may have changed something about their camera usage after said reddit thread. Can we get back on that topic?
What topic, the one I answered for you here? Neural Networks
The OP in the reddit thread is wrong, you just don't seem to want to accept that because of his huge improvement in the NN, while you seem to throw out the fact the NN is nearly 4x larger, and trained on a vastly larger dataset and much richer / deeper layers within. They haven't changed crap in terms of leveraging more cameras for driving.
Also, I'm assuming this is the OP? Those cameras are most definitely not covered all the way...
Tesla's AI-based Autopilot tested after owners tape cameras in experiment
Last edited:
I actually hadn't seen that video before since I only had his reddit post for reference, thanks. Now from the video it's clear that the amount of tape he placed on the front cameras was not sufficient. I can definitely conclude that his claims on reddit of AP2 only using the center fisheye camera are false.
Lets try to analyze his methology in said video.
We know, that the camera cluster should be arranged like this:
This is the tape coverage from the final test in the video:
With reference to my own gif of the heater wires in the fisheye camera I can conclude, that the fisheye camera fov is almost completely blocked. There is not much of the area below the bottom-most heater wire visible in the image.
But as for the other two front cameras... yeah... Main cam should still see a strip on the bottom and maybe on the right, narrow cam is only obstructed on the top right of its FOV.
With this I'm satisfied to say that we don't seem to see anything new. Still looks like main/narrow cam doing their stereo thing. Definitely matches what jimmy and the gang say. Case closed.
Definitely, we (I) did the same test on an earlier software version, and not taping all the way down affected the test. If the bottom part is not covered the camera is still able to see the road right in front of the car:
Partially covered:
Current HW2 Autopilot using 2 of 8 cameras * Testing Inside *
Fully covered:
Current HW2 Autopilot using 2 of 8 cameras * Testing Inside *
Many thanks, that's impressive.
When the weather changes, can you try the same route with low sun / strong shadows across the road?
Nice. Finally you joined our referal-beggers-club.
But House of pain was supposed to be played with the swerving firmwares, not 18.10.4.
Elon visits Cortica?
Tesla is reportedly in talks with Israeli AI vision firm as Elon Musk visits the country [Updated]
If something comes of this, let's hope it's better than Mobileye.
Tesla is reportedly in talks with Israeli AI vision firm as Elon Musk visits the country [Updated]
If something comes of this, let's hope it's better than Mobileye.
T-Will
Member
I think that was a beep warning just because of the car hugging the lane and the ultrasonics picking up the car.. Worth another look
John Beans
Member
Wonder how long it will be before someone notices that a Tesla engineer created a "Leguna Seca AP Easter Egg" to do laps (geo-fenced to the track, natch).
Maybe people even start teasing each other. "1:57? Dang, son, you know that car would actually go faster without you in it, right?"
Maybe people even start teasing each other. "1:57? Dang, son, you know that car would actually go faster without you in it, right?"
Similar threads
- Article
