Neural Networks

[mongo]

You lost me at unbiased.

Were you reading from bottom to top?

Analysis like how easy it is and how much data you would need to build a network with the same level accuracy as the Tesla network.

Interesting that they must already have networks at all levels of accuracy to tell how difficult the Tesla NN is to copy...

 

[Unpilot]

Were you reading from bottom to top?

Interesting that they must already have networks at all levels of accuracy to tell how difficult the Tesla NN is to copy...

kinda...with that guy I just sort of skim...it's not hard to see the bias.

 

[thimel]

An example of an algorithm that does bounding boxes is YOLO. You can see What is object detection? Introduction to YOLO algorithm - Appsilon Data Science | We Provide End to End Data Science Solutions for an overview of it. The article talks about x, y, height, width, classification, and confidence level. An example of the output could be something like (.85, 310,240,50,100,4) which means that it is "85% confident that a car (if 4 means car) is at coordinate 310,240 with a height of 50 pixels and a width of 100 pixels". Actually the output would be multiple entries like that - not just one.

A page thats a bit more technical but shows an example of how the driveable area can be highlighted is from the Nvidia site: Fast INT8 Inference for Autonomous Vehicles with TensorRT 3 | NVIDIA Developer Blog. Figure 3 shows the 19 classes they used and Figure 5 shows them overlaid on a sample scene - dark purple for road, yellow for traffic signs, etc. That example is like the one jimmy_d described in the Tesla NN where they started with an existing image model (VGG16) and built onto it. If I recall correctly, jimmy_d said Tesla's is based on Inception instead of VGG16 but its the same basic idea. So the output is a pixel by pixel mapping of what each pixel is. The Nvidia example is a 512×1024 image so the output could be a 512x1024 array where each entry is the number (1-19) of the class for that pixel.

When you say 'green driving path' I'm assuming you mean from the verygreen/damianXVI clips of driving in Paris, etc. So that would be just taking the dark purple section from the Nvidia example and overlaying it (i.e. alpha compositing so you can see through it) on the original image but with green coloring.

I have no insight into what Tesla is actually doing. Those are just simple examples of what *could* be done. Maybe jimmy_d or verygreen have actual output samples from Tesla's and can narrow it down more.

Thanks, this is just what I was looking for.

 

[Bladerskb]

This is a recommended and quite required watch if you want to post in this thread/forum.
It goes really in-depth about autonomous driving, hardware and software including pros and cons.
It goes step by step. its simply not just regurgitating "neural nets neural nets neural nets" like Tesla does.
It takes you through the entire platform, end to end.
Although it talks about neural nets, training, optimizing models and labeling, corner cases.

 

[Bladerskb]

Another good one.
Waymo's Neural Network use on their lidar and camera.

Neural Net talk starts at 31:00 to 54:00

 

[Bladerskb]

Out of curiosity what can I buy (car, electronics. etc) in the US today that has as much edge computing based neural network(s) running than a Tesla with V9? Obviously Facebook, Google, etc have absolutely massive neural networks, but that's not the same thing as they are up in the cloud.

As to the podcast I think it's important to understand that as a NN person, Jimmy is naturally going to be biased towards NN based approaches. We all have different perspectives, and different approaches to a problem we'd take. I'm a strong advocate for sensor fusion, and I don't think Tesla has enough with the current sensor suite.

First of all, Mobileye eyeq4 has L5 capable perception deep NN in them with well over 99.9% accuracy. Way more networks and detection than what's currently in v9 with better accuracy and efficiency (2.5 Tflops 3 Watts) You can find eyeq4 in several late 2018 model cars such as bmw, nio and other cars. Amnon claims that 2 million L3 cars will be released in 2019 running on EyeQ.

But let's ignore that. i want to ask you this question i asked someone else.

Elon said repeatedly that he was close to cross country demo in 2017 and said if it didn't happen in the end of the year it will be very close. 25 months later and yet still nothing. Was Tesla actually very close? Well let's say Tesla was very very close. Lets say they were 75% there.

That means their system could go over 2250 miles before disengagement.

THAT'S ASTONISHING!

Releasing that to the fleet would bring such exponential level of safety that has never been seen before.
Let's say they were only 50% there. That's still 1500 miles before disengagement. AP today can't even go 10 miles before disengagement.

Instead we get a useless NOA. if Tesla actually had a software that could go 1500 miles before disengagement they would release it in a heartbeat.

People complain about Demos but shouldn't you be asking yourself why hasn't Tesla released software that can do what's happening in these demos?

Isn't that the point of Tesla and taking the incremental agile approach? What others release as a demo progress, Tesla should be releasing as a production software. But we get the garbage that is NOA.

 

[mongo]

First of all, Mobileye eyeq4 has L5 capable perception deep NN in them with well over 99.9% accuracy.

Huh? So eyeq4 also has human speech/ voice command capability to handle public safety officer directions? (Needed for true level 5, any situation driving)

Way more networks and detection than what's currently in v9 with better accuracy and efficiency (2.5 Tflops 3 Watts)

More networks = more processing
More detection = more processing
More accuracy = more object types and possibly more over fitting
More types of objects = more processing

And I thought you haven't gotten access to Tesla's NN, so what are you using for comparison?

Instead we get a useless NOA.

Wow, useless. Seems a bit harsh for someone quoting Nissan's blurb on their delayed lane system.

if Tesla actually had a software that could go 1500 miles before disengagement they would release it in a heartbeat.

What is the disengagement rate of v9 on limited access freeways?

People complain about Demos but shouldn't you be asking yourself why hasn't Tesla released software that can do what's happening in these demos?

1. It's development SW that is not finished enough
2. It may be software that requires HW 3.0
3. What Tesla released demos are you even talking about?

Isn't that the point of Tesla and taking the incremental agile approach? What others release as a demo progress, Tesla should be releasing as a production software. But we get the garbage that is NOA.

How does Agile even fit with a base level NN development? You either have a generalized classifier or you don't. Once the basic driving side is good, then you can add on NN for stop signs, traffic lights, and such (isn't that what Tesla said they are doing?)

Again, why the hate for NoA?

 

[Matias]

First of all, Mobileye eyeq4 has L5 capable perception deep NN in them with well over 99.9% accuracy. Way more networks and detection than what's currently in v9 with better accuracy and efficiency (2.5 Tflops 3 Watts) You can find eyeq4 in several late 2018 model cars such as bmw, nio and other cars.

Honest question; why are we not seeing these level 5 cars in traffic?

 

[MTOman]

Elon said repeatedly that he was close to cross country demo in 2017 and said if it didn't happen in the end of the year it will be very close. 25 months later and yet still nothing. Was Tesla actually very close? Well let's say Tesla was very very close. Lets say they were 75% there.

That means their system could go over 2250 miles before disengagement.

That's making a lot of assumptions, he said they were close not that they could go 2250 miles without a disengagement. Maybe he meant close as in it could handle a majority of the highway driving but not the city driving on both ends of the trip, and perhaps some along the way getting from the highway to the superchargers. Perhaps he meant something else, we don't know, but he didn't say it could handle X many miles without a disengagement.

I'm not saying he wasn't wrong, he did say they would do a cross-country drive, but it doesn't help your argument when you put words in his mouth.

AP today can't even go 10 miles before disengagement.

If you can't go 10 miles today without a disengagement you probably should take your car to a service center. I, and many others on here, routinely travel dozens if not hundreds of miles without needing to disengage. In fact, the last trip I took I only disengaged when I was getting off the highway to charge or getting off the highway at my destination, I was able to go the 200 miles or so between on ramp and off ramp without having to disengage once. If your car can't go 10 miles without disengaging something definitely isn't right.

 

[Bladerskb]

Huh? So eyeq4 also has human speech/ voice command capability to handle public safety officer directions? (Needed for true level 5, any situation driving)

I have never in my life needed human speech / voice command to follow public safety directions, i follow their hand gestures.

And I thought you haven't gotten access to Tesla's NN, so what are you using for comparison?

Based on the unbiased analysis of the system by Verygreen. We know it doesn't detect general object obstacles, cones, debris, barriers, curbs, guard rails, traffic lights, traffic signs, animals, road markings, etc.

Tesla's SFS for example is basic, while Mobileye has 15 different categories and they might have added more. It will tell you whether the edge of the road is flat, whether its a curb, whether its a concrete wall, whether its a guard rail, and whether its a concrete barrier.

Even eyeq3 had a network that detected potholes, animals and road debris. Infact Audi is using it for Automatic Active Suspension adjustment.

They also have Networks to do things such as lane detection and what lane you are currently in, and upcoming lanes detection from afar. NOA could use a network like that as the way it handles things currently are primitive. right now it looks like they are doing some sort of algorithm downstream that barely works. Which is why it detects shoulders as a lane and also not seeing lanes which leads to missing exits and attempting to take them way too late.

For me, it identifies wide left shoulders as lanes when I am in the right lane (on a two-lane section), so it displays 3 lanes where there are two. It does this consistently. I think there is a somehow a tenuous connection between the map information and the lanes displayed. I think the behavior you see can be explained even if the car doesn't do anything with maps. If it sees three lanes to your left, and it knows you have an exit on the right coming up, it knows you have to move to the right. The closer you get to the exit the closer it wants to be to the right lane.

So my question is -- how does it handle exits/interchanges where you need to be in a specific lane which isn't leftmost or rightmost? Let's say that it does at it does for me all the time and recognizes the shoulder as a lane when you are not right next to the shoulder already, and you're on a three-lane road and you need to be in the middle lane. But it sees a 4-lane road, or maybe a 5-lane road if it recognizes both shoulders as lanes. What does it do? This gets way worse probably if you're on a 5-lane road and need to be in the middle lane, but it sees the shoulders as lanes if you're not in the leftmost or rightmost lane. It might think you're already in the lane you need to be in, especially if there's stop & go traffic and it can't see all the lanes all the way across because they're obstructed by cars.

ME also has a lane segmentation network that tells you what it lanes means and leads to. Lane expansion, merge lane, lane split, lane collapse, exit lane, etc. All of this is in Eyeq4 and in production TODAY!

I think I've figured out how NoA actually works. It doesn't really know much about lanes in absolute terms, like "this section of highway has 4 regular lanes and one entrance ramp acceleration lane which ends in 200ft, and I am currently in the acceleration lane", or "I am currently in lane 2 out of 4" or anything like that. I could be wrong, but based on my experience I think it's much simpler; it more or less knows only "I am in the rightmost lane because there's nothing to my right", or "I am in the leftmost lane because there's nothing to my left". And if it knows it's supposed to be exiting right in 1 mile, it just asks for lane changes to the right until there's nothing to its right.

Extremely disappointing and in my experience -- let me reiterate that this part is not conjecture but is my actual experience from using NoA on urban highways -- it is completely useless for urban highways and for commuting in heavy traffic. Never mind heavy traffic -- I've done two evening commutes that were well past rush hour and traffic was about as light as it ever is in an urban area, and it still couldn't handle it.

Wow, useless. Seems a bit harsh for someone quoting Nissan's blurb on their delayed lane system.

That remark is not from me, but from other reviews of NOA.

Day 4 with NoA, and it finally got an exit ramp "right" except then it immediately tried to smash me into a concrete barrier. So every other exit I've tried to let NoA handle, it has waited until the exit is halfway behind it before putting on the turn signal and then getting into the lane. This morning for the first time it recognized the exit immediately, as the lane was appearing, and started moving into it even before it was a full-width lane. "Great!", I thought at first, "it's finally doing this more or less the way it's supposed to!" Except there was no shoulder on this exit ramp, only a concrete barrier. And it zoomed into the lane before it was full-width -- not gradually, following the right lane line as it expanded, as a person would have, but suddenly as if the whole lane was already there. So I didn't have long (0.5s maybe) to be impressed before I was taking over to avoid hitting that barrier.

I should also note that, as usual for exit lanes, it did not ask for confirmation before executing this maneuver. If I hadn't been vigilant with my hands on the wheel and my eyes on the road, I would likely have bounced off that barrier. I should also mention that it was dark and rainy at the time (as it often is during my commte...)

Stay safe people.

What is the disengagement rate of v9 on limited access freeways?

• EAP is fine but still tries to take exits if I'm in the right hand lane. Very surprised this hasn't been fixed since there have been so many complaints about it.
• NoA for me is pretty much a disaster. I'd call it alpha not beta based on how poorly it has worked.
• Several times NoA wants me to switch to the faster lane when I'm within a mile of the exit with the right hand lane already slowing down. Have had to manual move to the right hand lane.
• Several of the exits have fairly tight lanes and to avoid hitting the standard lane cement barriers I took over. Not sure if I would have hit the barriers but too close for me.
• Often when I've merged on the highway there was too much acceleration and got way too close to the car in front of us.
• In general I've had to take over pretty much every time and cannot imagine what would have happened if Tesla had not required confirmation.

• Does not understand cars merging into the lane at all (this is kind of a big deal)
• Less confident at lane keeping as it gets darker outside
• Challenged by construction areas with barrier walls (likes to hug too close for comfort)

For me, the lack of perception for any and all lane merging is a big minus (you'll either need to proactively stay toward the left-most lane, or otherwise constantly toggle the feature on and off).

I want to like Navigate on Autopilot, but I just can't do it yet:
- One specific freeway has two entrances, each with two lanes. Those four lanes feed into one lane to actually get on the freeway. The car has no real idea how to handle this, so it's pretty unsafe to actually give it a chance and see what it'll do.
- It still aborts lane changes part-way through, seemingly at random.

It's a decent first attempt, but it absolutely deserves the beta tag that it's wearing.

Day 3 commuting with NoA... it has still never once actively merged onto a highway. If I don't take over, it just cruises all the way to the end of the acceleration lane and then swerves over into the highway lane at the last minute, as the lane is ending, without ever signalling or prompting for a lane change. For exit ramps, as I said above, it waits until the deceleration lane is half gone before signalling and taking the exit, which if the deceleration lane is short (most are around here) is a real problem.

Last night I got to try it on several highway interchanges because I had a longer drive. It was 1 for 4 on those interchanges; the other 3 times I had to take over.

1. It has trouble to determine I am on the right most lane which leads to an exit that's only half way to my office. When it figures out, it's already too late and I have to disengage and change the lane myself.

2. When it came to the right exit, it's again too slow to tell me and fortunately there isn't any traffic at the moment, otherwise I am traveling too fast and missed the exit.

So far not overly impressed with Navigate on Autopilot. I engaged autopilot while on an onramp to the expressway. There was a car comfortably behind mine to the rear left, and another in front of me exiting (the onramp continues parallel to the highway and eventually converts to an offramp for the next exit). Despite signaling, the car was not going to change lanes onto the expressway without me intervening. It was very hesitant, slowing down, and speeding up seemingly at random, but not committing to a lane change. Once on the highway, I had my speed set to 70 MPH. Driving behind a truck going 50 in the far right lane almost 3 miles from my exit in light traffic, and the car never suggested a lane change to overtake the truck. I had the speed based lane changes set to aggressive. Instead I initiated a lane change myself. The car did suggest a lane change back to the right lane as I approached my exit, but then immediately began slowing down on the high way, dropping my speed below 50 MPH, despite my speed being set at 70 MPH and no vehicles in front of me and being approximately .5 miles away from my exit. I had to take over to avoid getting rear ended, re-engaged auto pilot, but the system then disengaged navigate on auto pilot before taking my exit.

So now I have two morning and one evening commute under by belt. The first morning was a disaster as described above. The evening and second morning were both in much lighter traffic and were less terrifying but still mostly useless. It is 0 for 3 on taking any action whatsoever to merge from an entrance ramp onto the highway.

1. It's development SW that is not finished enough
2. It may be software that requires HW 3.0
3. What Tesla released demos are you even talking about?

Again Elon boasted that AP2 chip was more than enough for Level 5 FSD and grandstanded that he could do cross country with his eyes closed and promised FSD feature to be released in early 2017. Have you forgotten all of this? How convenient!

How does Agile even fit with a base level NN development? You either have a generalized classifier or you don't. Once the basic driving side is good, then you can add on NN for stop signs, traffic lights, and such (isn't that what Tesla said they are doing?)

Again, why the hate for NoA?

Its not the NN that is the problem (when it comes to highway autonomy for the most part), its the motion planning and control algorithm. How is it that you people never seem to be able to differentiate the two?

 

[rnortman]

What is the disengagement rate of v9 on limited access freeways?

By now it does extremely well at basic lane keeping and TACC on well-marked highways without traffic that cuts you off semi-aggressively. It does pretty well at auto lane change. (I mean one you initiate, not Nav on AP.) It still cannot handle merging, either you merging into traffic or traffic merging into your lane. This is the most common reason that I take over. But if you're on a long stretch with nobody merging aggressively and no construction, and you're not going to fast for the curves (or, faster than it likes), then it does extremely well now. Phantom braking is also still bad; possibly worse in V9 than in the later versions of V8. (It's hard to say as my sample size is relatively small.)

They still have a long way to go to even call this L3 on those highways. It's a very good L2 system on the highway by now, I would say. But being L2, it requires constant vigilance.

The other very common reason I take over is when I see traffic well ahead of me slowing down or stopped. I know that TACC will react too late for comfort and so I disengage preemptively and let the regen braking do its thing. Their cameras really ought to have enough range to detect this, if there aren't hills/valleys/curves involved, but it seems not to look ahead very far still -- certainly not as far ahead as a human driver does.

 

[rnortman]

Its not the NN that is the problem (when it comes to highway autonomy for the most part), its the motion planning and control algorithm. How is it that you people never seem to be able to differentiate the two?

Because a great many people believe in this idea that the NN ("Software 2.0") will be able to handle motion planning and control. You just need a faster chip and another 10M fleet miles, and suddenly you'll have an end-to-end NN L5 system.

We shall see.

 

[lunitiks]

We shall see.

@AnxietyRanger is that you?

 

[jimmy_d]

Jimmy, can you provide Blader the metadata you are looking at? Would be cool to see his interpretation of it added to the mix.

It's not my data to give out. That said - there seem to be a number of people who have the ability to acquire it on their own and are happy to share it. Anyone genuinely interested can acquire it if they ask around politely.

 

[jimmy_d]

I have a NN question. The answer is probably not specific to Tesla’s NN, so anyone who knows how this works can chime in.

What form are the outputs of the NN in. In particular are the bounding boxes (of cars and pedestrians) given as XY coordinates of two corners? There can be a variable number of bounding boxes. How is that handled? How is the type of vehicle (car, truck, pedestrian) given?

Is the green driving path simply given as a rectangular pixel grid with the same resolution as the input?

I've detailed that stuff in posts near the beginning of this thread, here's an early one: #7

Bounding box outputs from NNs aren't generated as drawn boxes, generally. The output is derived from an image map that breaks the image into sections, perhaps as small as a single pixel but usually larger, with each section providing a candidate category for the object it is associated with in addition to estimates for the top, bottom, left, and right extents of the object. The bounding box for that object is generated from all of the section estimates, often by computing an average. This is the YOLO style approach. There are other ways but they aren't generally considered applicable to real time applications like driving. The number of bounding boxes cannot exceed the number of image sections and is generally much smaller but, for instance, having hundreds of bounding boxes in a single image is certainly possible.

Semantic segmentation maps generally provide a category for each pixel in an image but without the bounding box estimates. They are useful for sectioning an image into areas that have highly variable boundaries but which nonetheless need to be identified in order to generate context. Examples of this would be stuff like: driveable area, pavement marking, sky, lawn, obstruction, etc.

The shape of the network doesn't tell me what categories are getting generated - the label itself is just an integer. But from the work of @verygreen and others it's safe to say that at least some of the categories are: pedestrian, cyclist, motorcycle, truck, minivan, car, large truck, bus, pavement marking, stop sign, traffic light, driveable area, and pole. There are probably a lot of other categories as well but these are things that get exposed to some extent.

 

[jimmy_d]

Does anyone have experience with or knowledge of using GPUs on AWS? I recently read that Google Brain used ~45,000 GPU hours to use AutoML/Neural Architecture Search to create a neural network architecture called NASNet. I looked up AWS pricing for GPUs and it looks like it's 40 cents per hour. So, 45,000 GPU hours would only be $18,000.

Even for a small startup, that seems reasonable. For a big company, you could afford to pay much more. If you wanted to use 1000x as much computation as Google Brain, if you wanted to use 45 million GPU hours, it would cost $18 million. For Tesla, that feels like a drop in the bucket.

Does that sound right? This cost seems ridiculously low.

Interesting also when you look at Efficient Neural Architecture Search (ENAS), which is attempting to bring the computation cost of Neural Architecture Search (NAS) down by 1000x. If ENAS can achieve equally good results as the NAS that Google Brain used, then with 45 million GPU hours you could do 1,000,000x as much search as Google Brain. Crazy.

Say Tesla really wanted to go nuts with AutoML and spend $180 million. That wouldn't be unfeasible; Tesla could still stay profitable and cash flow positive if it spent another $180 million on R&D in one quarter. With regular NAS, it could do 10,000x as much search as it took to find NASNet. With working ENAS, it could do 10,000,000x more.

Unless I'm getting the actual AWS pricing wrong. So please let me know!

Incidentally, this is why I want mad scientist Elon to stay in control of Tesla. Or at least for Tesla to have a Board that gives Elon the freedom to run the company. I have the feeling that many of the Boards of public companies (outside of the tech world, at least) would lack the imagination to approve of this kind of spending on a mad science project. Yeah, Elon is crazy, and I goddamn hope he stays that way.

AWS pricing varies a lot depending on your job constraints, machine type, interconnect and storage requirements, but the pricing you cite is in the ballpark.

 

[jimmy_d]

Thanks! So helpful.

Funny that there is a chip called the Nvidia Tesla P100. Did we run out of names?

Okay, here's another way to do the math. Google Brain used around 500 Nvidia P100s, the most expensive version of which costs around $10,000. So 500 of them costs $5 million.

If I'm getting my facts right, Google Brain completed its Neural Architecture Search within 4 days. If you were willing to wait 28 days (and why not? what's the rush?), you only need to buy 1/7th as many GPUs. So 72 would be enough. If you bought 720 GPUs for $7.2 million, you could do 10x as much computation as Google in 28 days. If you bought 7200 for $72 million, you could do 100x as much.

This is conservative because I rounded some numbers up, and used the price of the older chip when it first came out (I think).

Alternatively, if you use AWS pricing and (again, conservatively) treat 1 P100 as equal to 1 V100 hour, the cost of 4.5 million GPU hours (100x what Google did) at $3.06/hour would be $13.8 million.

Come to think of it, could you even use AWS for this? Would you need one big custom instance with like 100 GPUs? Can you split up the NAS task between multiple instances? Is this all solved using virtual machines?

You might want to consider that the final training run for a job is a small fraction of the total time used to develop a neural network. Often you run many variations of a training run over a long period of time before you find a formula that works and are able to generate the network you want. The advantage to having a lot of machines is that you can get faster turn around on your tests. If you have to do dozens of trial runs before you get what you're looking for and each one of those takes a week or more then the calendar time needed becomes excessive.

Large organizations often have a pool of resources that a bunch of researchers share for running big jobs and smaller dedicated machines (think 4 to 16 GPUs) for individual researchers. You can generally work out ideas for how you want to go about accomplishing something on a small machine. You go to the big machine once you have a pretty good plan and want to throw a lot of resources at it.

 

[strangecosmos]

Yo Jimmy, what do you think of the hypotenuse that AKnet v9 was created partially using AutoML / Neural Architecture Search (NAS)? Not the stuff like processing two frames instead of one, which if you’re right about seems hand-coded.

But there’s no reason that a neural network architecture couldn’t be created using NAS, and then modified through hand engineering, right?

NASNet was optimized against the 50,000 training images in CIFAR-10 and exceed any hand-designed neural network at top-1 accuracy on ImageNet. It seems like Tesla could have a library of images from HW2 cameras that’s in the millions. I’m really into the idea of Tesla optimizing against a multi-million image dataset using Efficient Neural Architecture Search (ENAS), and possibly also doing more search, i.e. using a larger search space.

This would be in line with Karpathy’s Software 2.0 philosophy, and he mentioned NAS in his TRAIN AI talk.

Is there any reason why Tesla wouldn’t do this?

 

[strangecosmos]

You might want to consider that the final training run for a job is a small fraction of the total time used to develop a neural network. Often you run many variations of a training run over a long period of time before you find a formula that works and are able to generate the network you want. The advantage to having a lot of machines is that you can get faster turn around on your tests. If you have to do dozens of trial runs before you get what you're looking for and each one of those takes a week or more then the calendar time needed becomes excessive.

Large organizations often have a pool of resources that a bunch of researchers share for running big jobs and smaller dedicated machines (think 4 to 16 GPUs) for individual researchers. You can generally work out ideas for how you want to go about accomplishing something on a small machine. You go to the big machine once you have a pretty good plan and want to throw a lot of resources at it.

But this is when you’re hand designing the network, correct? Wouldn’t it be different if you’re using AutoML / Neural Architecture Search to design the network via algorithm?

 

[jimmy_d]

Yo Jimmy, what do you think of the hypotenuse that AKnet v9 was created partially using AutoML / Neural Architecture Search (NAS)? Not the stuff like processing two frames instead of one, which if you’re right about seems hand-coded.

But there’s no reason in principle that a neural network architecture couldn’t be created using NAS, and then modified through hand engineering, right?

NASNet was optimized against the 50,000 training images in CIFAR-10 and exceed any hand-designed neural network at top-1 accuracy on ImageNet. It seems like Tesla could easily have a library of images from HW2 cameras that’s in the millions. I’m really into the idea of Tesla optimizing against a multi-million image dataset using Efficient Neural Architecture Search (ENAS), and possibly also doing more search, i.e. using a larger search space.

This would we in line with Karpathy’s Software 2.0 philosophy, and he mentioned NAS in his TRAIN AI talk.

Is there any reason why Tesla wouldn’t do this?

The basic dataflow in aknet_v9 is just inception V1. Tesla might be using AUTOML ideas for hyperparameter search but they haven't done it with any of the camera network architectures for which I have seen detailed architectural data. As for why they would or wouldn't be using this stuff - from public discussion of Tesla's development it seems that their primary focus has been on creating a high quality training corpus. In that situation you don't need to change hyperparameters very often so automating that aspect of development is less valuable. Network search and hyperparameter search are incredibly useful in research and early development efforts but probably a lot less so for moderately mature product development efforts where your architectural constraints are more likely to be driven by the hardware you have to work with. Google's NMTS architecture was exactly the maximum that could be fit efficiently into a single computer chassis. That wasn't a cosmic coincidence, it was because that was the most efficient overall approach given their resources. Likewise Tesla's network is likely to be at exactly the limit of what HW3 is capable of providing and will be selected from the type of network architecture that their chip will be most efficient at processing.

Research and product development are different things.