NN vs Computer Vision for driving

[BigD0g]

Interesting paper and read out of MIT based on Tesla's vision approach compared to an end to end trained nueral network:
https://arxiv.org/pdf/1710.04459.pdf

 

[S4WRXTTCS]

Just a minor niggle.

It's not the Tesla vision approach they're comparing the end-to-end neural network with. They're comparing it with the MobileEye based first generation Autopilot.

The Tesla vision approach is HW2

 

[BigD0g]

Just a minor niggle.

It's not the Tesla vision approach they're comparing the end-to-end neural network with. They're comparing it with the MobileEye based first generation Autopilot.

The Tesla vision approach is HW2

Good catch!

 

[S4WRXTTCS]

It's exciting that people are using an NVidia TX2 for research such as this paper. I have a few at work, and one at home for some projects I'm doing.

I had no idea that this is what the MIT people were doing. Awhile ago they asked for Model S owners who would volunteer, but I believe that was for a different project. I imagine some of them carried over onto this one.

I would have volunteered, but I don't live in Boston.

I might play around with this just to experiment.

 

[croman]

So would Tesla implement a CNN just for arguing with their standard programming? I like the adversarial approach that this system takes before it requires human input.

 

[jimmy_d]

My first reaction was surprise that they have 6500 disengagements in 420 hours of driving data. That's more than one every 4 minutes.

 

[jimmy_d]

So would Tesla implement a CNN just for arguing with their standard programming? I like the adversarial approach that this system takes before it requires human input.

Building networks of networks is a pretty standard technique for improving performance - you see it used all kinds of places. Generally you have a family of systems trying to make a decision and you train a simple NN to sit on top of the whole bunch and make an overall decision based on the input and the 'votes' from the various subsystems. The top NN learns who to trust in what situations and optimizes the overall decision making.

Once upon a time researchers would try to synthetically generate adversarial opinions by systematically varying the training of various subsystems on a common pool of data. Dropout, which has become one of the most powerful regularization techniques in use, was developed by Geoff Hinton as a kind of mathematical extreme version of this. And of course now we've come full circle with GAN (generative adversarial networks) where you train a network to do some kind of task and you simultaneously train another network to mess with the first one and cause it to slip up. The first network gets better and better at it's job because the second network is constantly getting better and better at screwing with it.

I've wondered what the self driving car version of a GAN network might look like. The second network (call it the loki network) would be constantly trying to invent tricks to mess up the driving network. So in a sense it is automatically looking to create corner case situations, and then the driving network gets better by learning to deal with the new corner cases.

 

[croman]

@jimmy_d Loki network is very fitting. Thanks for the informative post.

 

[BigD0g]

@jimmy_d very insightful and informative thank you for the post.

 

[calisnow]

My first reaction was surprise that they have 6500 disengagements in 420 hours of driving data. That's more than one every 4 minutes.

That's fantastic performance - you don't think? 420 hrs is hardly any.

 

[calisnow]

That's fantastic performance - you don't think? 420 hrs is hardly any.

I wasn't being sarcastic @jimmy_d - 420 hrs is a very small dataset isn't it - in terms of finding and learning disengagement events.

 

[jimmy_d]

I wasn't being sarcastic @jimmy_d - 420 hrs is a very small dataset isn't it - in terms of finding and learning disengagement events.

I thought it was funny that you were seeing it in the reverse way I was: I was thinking of it as a pretty crappy driving experience and you were thinking of it as a barely adequate machine learning database - if I understand you correctly. It's funny to me because your point is a very good one and one I hadn't considered in my original comment.

But along those lines - yeah, I'm totally shocked at how little training data it takes to get something basic to work in this arena. Lots of projects manage to get something that can do basic driving with less than 100 hours of training data. That just doesn't seem like a lot to me as a human driver.

I guess when you think about it 420 hours of video is a lot of raw data and probably means you don't have to do much regularization on a network with 100 million parameters or so, which is probably the scale of the nets being used in these systems, but somehow it still doesn't seem like a lot. Although I guess it's true that humans can earn a drivers license with even less time behind the wheel. I loved seeing the youtube videos of people using the comma system because I'm imagining that if an effort as small as comma can get something working that well that the future for these systems is really bright.

Of course I understand that 100 hours of training stuff is far from being good enough for a commercial product. Lots and lots of testing and tuning are needed to get something out the door, which is a big part of why Mobileye's system is so good even with it's limitations.

Still, I'm really hopeful that pure NN's with serious hardware backing them up are going to show us really impressive gains in performance over the coming months and years.

 

[croman]

In IL you just needed 25 hours of driving time before you could take the driver's license test. I'm not sure if you'll pass, but that was all that was required. AP2 certainly wouldn't pass but at least it would probably pass the parallel parking portion of the test.