Neural Networks

[jimmy_d]

Car manufacturers usually buy competing cars, take them apart, compare every detail of them and see what they can learn from them. This is very standard. In this case they will learn the sensor suite Tesla is using, the ethernet network layout they use, the compute hardware, the neural network input, the neural network model and the weights. So they could probably spend a year to rebuild the same in their own car using the same weight, but that will be tricky as the weights are very tuned to the specific system. They could try to duplicate the data Tesla has also and start to train their own weights using the same model, but that will take a lot of time. It is very hard to be a copy cat, you will always be behind and the engineers will not feel very creative and look for options. And customers are willing to spend less on it. And there might be some legal actions Tesla can take to defend themselves against this.

Likely what they will do is learn what Tesla has done, then bring it into their design discussions. And they don't have access to the same data gathering capabilities that Tesla has so likely they will have to find a different approach and maybe network architecture tailored to their data. And the resistance to learning from what Tesla is doing can be heavy among the old companies, I say this from experience having held presentation to a car manufacturer about what I thought Tesla's strategy was. Maybe the Chinese and the Silicon Valley companies are more open to learning from Tesla's solutions.

Fwiw, the network is not camera agnostic. It is maybe somewhat calibration agnostic, but is likely very much CMYK dependent and probably dependent on many other things also.

Interesting - it sounds like you're saying part of the calibration process is possibly color calibration. Is that right? I'm not familiar with how much color variation affects NN performance or how much color variation there is in commercial cameras. Because NN's need to be relatively immune to variations in lighting (cloudy day, sunset, street lamps) I would have thought that absolute color calibration wasn't very important. But of course if commercial camera variation is really large then maybe that gets swamped.

Do you have any info in that area?

 

[Vitold]

I don’t see what would be a point of stealing Tesla non right now. It’s half baked and guaranteed to change.

Biggest value to others is for use it as a benchmark to their systems. Basically given Tesla’s openness at the moment they setup a baseline for others to aim for and hopefully exceed. The problem for reverse engineers is that Tesla AP is a moving target and will be for years to come also reliance on NN at higher levels ,as it was pointed ,out will obfuscate it naturally.

 

[fmonera]

But it’s not like Tesla is giving away valuable intellectual property by not encrypting this stuff. Another company can’t just copy Tesla’s neural network architecture by hacking a Tesla. Right?

On our cars we just have the Neural Network and the matrix of weights (a few thousand of numbers).

The source code (1.0) of a Neural Network is just a few hundred of lines (or 2 lines in python) and it is always almost the same. There is no real value on that.

The matrix of weights is a snapshot of the brain at a given moment. It cannot be used to improve or learn. Is like a brain that cannot learn anything else. For example, it would be useless for another brand of cars (unless they use the same sensors and cameras on the same position) and, more importantly, they cannot use that matrix of weights as a head start to make a pirated copy of autopilot and improve from that.

The code (1.0) needed to train the NN is not on our cars, although, again, it provides almost no value. Nothing revolutionary here.

The real value of a Neural Network is the examples database which is used to train the network. And that is not on our cars, but on the mothership at Tesla.

 

[S4WRXTTCS]

I’m not sure about this. Ownership of car sounds irrelevant to discussing progress of autonomous vehicles. Ownership may even be misleading if currently owned car is not representative of what companies are working on and what can appear soon on the market?

Without being an owner how can you measure progress? How can you see what it did yesterday versus today?

We're a long ways away from autonomous full-self driving, and so the behind the wheel experience is paramount in knowing whether any given update is a step forwards or a step backwards.

I'm not saying that non-owners can't offer valuable insights, and so forth. I simply question non-owners that try to involve themselves so much without really knowing what it's like to use these systems.

If I joined a BMW forum to talk about the new X5 with the EyeQ4 system I wouldn't start talking smack based on things I read on the internet. Instead I would mostly listen to learn about the system, and when I had a chance I'd rent one to try it out. To get at least a few thousand miles of behind the wheel experience.

 

[electronblue]

Without being an owner how can you measure progress? How can you see what it did yesterday versus today?

From whatever sources that may be available and relevant? I understood Autonomous Vehicles being on Tesla Motor Club’s General Forum so not really Tesla related. First hand data is useful when available for sure but ownership tends to skew perspectives too so it takes all kinds?

 

[strangecosmos]

We're a long ways away from autonomous full-self driving

I think there are (at least) three exponential trends in autonomy:

1. neural network size (enabled by more powerful embedded compute)

2. availability and collection of training data

3. backend compute (useful for simulation, training, AutoML, etc.)

All three of these things are increasing something like 10x every 1-2 years, much faster than Moore's law's 2x.

When core aspects of the technology are changing exponentially, I think it makes it hard to predict with your gut what the trajectory of future progress will look like.

The unanswered question is whether neural network performance and hence driving performance will increase exponentially along with neural network size, training data, and backend compute, or whether we'll run into quickly diminishing returns. This is something that Karpathy's team can probably measure right now, but the rest of the world is still in the dark.

I think the most rational/empirical position right now is agnosticism on the timeline. Unless you have inside information at Tesla, Waymo, et al., I just don't see how you can predict whether fully autonomy will happen in 2019 or 2025.

I understand that people have strong gut intuitions, but why would gut intuitions tell us about the truth? If gut intuitions are a reliable guide, why are people's gut intuitions on this question so wildly divergent?

The deployment of HW3 will be a major test that will give us new information. How much driving improvement does 10x more embedded compute (and a commensurately larger neural network) cause? Is in the improvement in driving closer to 10x or 10%?

There may be a lag with HW3. With perception tasks, Tesla can train and test the HW3 neural networks offline, using their accumulated video data. So I don't see why the HW3 neural networks wouldn't do better on perception on day one. But if path planning uses a neural network (still unclear on that), you may need the neural network to give you its output in a real world driving context before you can train it and test it at scale. So that may take time.

If, say, six months after the launch of HW3, driving performance is still only 10% better than with HW2, that would be evidence of relatively slow progress, where performance increases 10% every 1-2 years despite the inputs increasing 10x. So it could take many years before we see full autonomy.

If six months after the launch of HW3, we see a sudden step change in performance and it's several times better than on HW2, that is evidence of a steep, AlphaGo-like trajectory of improvement from subhuman performance to superhuman performance in just a few years.

Right now we are in an interesting moment of ignorance when we don't know which one it is. The scales will tip one way or the other within about a year, and we don't know which way. I know which one I hope it will be, but I have no way of knowing which one it will actually be. And I don't see how anyone who isn't under an NDA has a way of knowing either.

 

[S4WRXTTCS]

On our cars we just have the Neural Network and the matrix of weights (a few thousand of numbers).

The source code (1.0) of a Neural Network is just a few hundred of lines (or 2 lines in python) and it is always almost the same. There is no real value on that.

The matrix of weights is a snapshot of the brain at a given moment. It cannot be used to improve or learn. Is like a brain that cannot learn anything else. For example, it would be useless for another brand of cars (unless they use the same sensors and cameras on the same position) and, more importantly, they cannot use that matrix of weights as a head start to make a pirated copy of autopilot and improve from that.

The code (1.0) needed to train the NN is not on our cars, although, again, it provides almost no value. Nothing revolutionary here.

The real value of a Neural Network is the examples database which is used to train the network. And that is not on our cars, but on the mothership at Tesla.

I don't question that biggest value is the database to train the network. But, to me there is value in the network itself. Over the last 5 years we've had significant gains in speed/accuracy of neural networks of publicly available networks, and I imagine the same is true for private ones.

There is also the need for hardware acceleration, and usually requires customizing layers to get it to work with the vendors tools for acceleration (like TensorRT if you're using Nvidia hardware). To me the network would be valuable IP especially for large networks that would be worth protecting.

BUT, and there is a but.

In the AI community there seems to be a STRONG desire for openness. In fact one of the most awesome things about studying AI is you can generally access most of the research papers for free. There is also the blackbox fear in that we want the ability to verify that this blackbox does what it's claimed. This is especially important with self-driving cars.

How do I know the rear-view side cameras, and the neural network can always detect vehicles? The community really should have access to test/validate these blackboxes independently from the manufacture.

With Tesla they're kind of in a weird spot where hackers can get the prototxt of the network (showing the layers), and the Model(weights). But, they don't have any right to post it so I'm not sure we can really say its independently verified if only "the people invited to Paris for the clubhouse meeting" have had eyes on it.

To my knowledge no one has independently run inference on a Tesla provided neural network. Hopefully I'm mistaken about that as that would be exciting.

 

[heltok]

Interesting - it sounds like you're saying part of the calibration process is possibly color calibration.

Camera calibration is the process of estimating intrinsic and/or extrinsic parameters. Intrinsic parameters deal with the camera's internal characteristics, such as, its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world.

As for color, the neural network just get a lot of zeros and ones and from this somehow figures out how the world looks like. If it is trained on CMYK and gets an RGB image it will likely be somewhat confused.

 

[ZeApelido]

Agree.

There are certain aspects to the use of NNs in software which is a kind of IP protection that is very powerful in and of itself. First - NNs are famously very black box in that they cannot be precisely understood in any straightforward way from an examination of their structure or behavior. This is a disadvantage in many ways but one useful feature (useful to some people, anyway) is that reverse engineering them is practically impossible. This contributes to IP protection when considered alongside another feature of NNs, which is that because a production NN is the product of a system that automatically generates the NN from a large database of training data and is quite sensitive to the details of the system that produces it you can't reproduce it without a complete and detailed set of all the data that went into producing it. What this means is that you can distribute your NN all you want and nobody is going to be able to extract any useful feature of it's overall behavior which is smaller than the complete NN.

Of course anybody who can use the NN exactly as is can just take a binary copy of it and use it directly. But if the target use differs in any significant way, no matter how minor, you cannot get much value from taking the NN. For instance, if you built another car that was mostly like a Tesla but not *identical* to a Tesla in terms of it's sensor suite, sensor placement, and control dynamics then the NN would probably not work very well. And you cannot modify the NN without access to all the data, tools, and settings that are used to produce it.

Of course there are some corner cases where you can train a modifier NN to take the output of an NN and adapt it to a slightly different situation, but even this is a major undertaking in the large majority of cases. You would still need an enormous curated database of training examples for your target vehicle - it would just be somewhat smaller than the database needed for the original network.

So in a sense, just using an NN in your design gets you a very useful level of IP protection so long as you keep tight control of the training data and training parameters. Since all that stuff lives in a well secured data center and never leaves your facility it's not particularly vulnerable to hacking. This would not be like hacking into google and trying to steal a file - it would be like hacking into google and trying to steal the contents of the entire data center - not impossible but quite infeasible.

As an aside - this is the method that Comma AI is using in their business model. They have open sourced their software and they give away almost all the details of their system except for the NN's production system. By keeping control of the development details and training database for the NN they can ensure that they will control their ecosystem without having to use any encryption or other form of DRM.

The problem is if the NN is so sensitive to many aspects that other manufacturers cannot replicate, it will also be sensitive to deviations in quality of those aspects in Tesla vehicles themselves...

 

[Soda Popinski]

If it is trained on CMYK and gets an RGB image it will likely be somewhat confused.

Why on Earth would anybody train a NN for image recognition from digital cameras on CMYK images? As far as I know, unless you're dealing with a specialized camera (like for use in Mars probes), your color digital sensor uses something like the Bayer array, which is an RGB filter. CMYK is a color space used in 4 ink / pigment offset printing. The K is black to compensate for the fact that C+M+Y inks should, but doesn't make a perfect black (inks and pigments are subtractive, unlike light / RGB, which is additive). In general, to generate a CMYK image, you need to start with an RGB image from a digital camera or scanner, and convert that into CMYK.

That said, there are various RGB color spaces, giving different gamuts, such as Adobe RGB, and sRGB. I would think the biggest issue with color calibration on NN would be white balance. Something our brains do automatically is make the dominant light we see "whiter." This is why we don't notice fluorescent lighting (the old style) is green and incandescent is very amber. Back in the days of film, you needed to buy color balanced film that would compensate for this difference so you get proper looking flesh tones. In the digital age, the raw image needs to be processed to account for this, otherwise a green light might not look quite green.

 

[heltok]

Why on Earth would anybody train a NN for image recognition from digital cameras on CMYK images? As far as I know, unless you're dealing with a specialized camera (like for use in Mars probes), your color digital sensor uses something like the Bayer array, which is an RGB filter.

I might have been misled. Someone said Tesla was using GGGR cameras. In another place, can't find it now, I wrote that I had been told that Teslas was using GGGR. Then someone there corrected me and said they had switched from GGGR to CMYK. Now you say Bayer, which is RGBG right? Another electrek post said they were using black&white as input.

 

[electronblue]

@heltok Autopilot 2 uses CCCR cameras so black/white/red. Autopilot 2.5 adds blue.

 

[mongo]

I might have been misled. Someone said Tesla was using GGGR cameras. In another place, can't find it now, I wrote that I had been told that Teslas was using GGGR. Then someone there corrected me and said they had switched from GGGR to CMYK. Now you say Bayer, which is RGBG right? Another electrek post said they were using black&white as input.

They weren't saying Tesla was RGB in that post.
Tesla was Red, grey, grey, grey , now it's red, blue, grey, grey.

2.0:
Elon Musk on Twitter

we could enable that, although the cameras use grey, grey, grey, red detection, so it would look a little greyish red

HW2.5:
HW2.5 capabilities

 

[Soda Popinski]

I might have been misled. Someone said Tesla was using GGGR cameras. In another place, can't find it now, I wrote that I had been told that Teslas was using GGGR. Then someone there corrected me and said they had switched from GGGR to CMYK.

I suppose a CMYK sensor would involve cyan, magenta, yellow filters, and an unfiltered 4th sensel (like a pixel, but sensor) to handle black/white. It looks like 15 years ago or so, Sony was expected to release a CMYK sensor, but I'm not sure if anything came of it. Having a dedicated black/white sensel was said to help reduce noise, which would increase sensitivity. So you might be right.

I have no idea what type of filter Tesla is using, but I do also recall hearing it wasn't a standard RGB / bayer filter. Based on the hacked images, it does look like at least the side cameras aren't full color. Maybe the hackers know the sensor type.

 

[Bladerskb]

If I joined a BMW forum to talk about the new X5 with the EyeQ4 system I wouldn't start talking smack based on things I read on the internet. Instead I would mostly listen to learn about the system, and when I had a chance I'd rent one to try it out. To get at least a few thousand miles of behind the wheel experience.

The reason why i'm not going BMW again for my next car is their lack of communication. There are dozens of ppl who are trying to decide whether to buy a car now or wait till 12/2018. But they can't get a straight answer even from corporate whether a car purchased now will be updated with the hands free traffic jam pro. That's beyond horrible. People think i'm anti-Tesla but i have said it time and time again, alot of auto companies are gonna go bankrupt and disappear after the next revolution, just like what happened with the phone.

 

[lunitiks]

the next revolution

???

 

[heltok]

They weren't saying Tesla was RGB in that post.
Tesla was Red, grey, grey, grey , now it's red, blue, grey, grey.

2.0:
Elon Musk on Twitter


HW2.5:
HW2.5 capabilities

Interesting, thanks!

 

[Vitold]

With the absence of green filter, how well can yellow lines be detected with red,blue,gray sensors?

 

[mongo]

With the absence of green filter, how well can yellow lines be detected with red,blue,gray sensors?

Depending on response spectrum (filters), green = grey-red-blue

 

[Bladerskb]

I think its good for me to post this. For all the hoopla about NN. Looks like i was right again. the driving policy algorithm is doing simplistic calculations.

It sees a lane that looks like an exit lane and then it takes it. Turns out its an emergency shoulder stop lane.

This is a problematic and shows what happens when you don't have a HD map.
It also showcases the primitive of the driving algorithm.
it also showcases what happens when you don't have lane semantic and strong perception NN.

Basically the algorithm right now is, when in the right most and you need to take an exit. if another lane shows up, change lanes immediately. So it sees the dashed lane markings and automatically changes lanes.

It also says the NN which has been praised is primitive because it uses lane markings to determine lanes instead of using semantics path delimiter and strong perception (ala mobileye).

Hence it shows shoulders as lanes...etc

rnortman said it weeks ago and no one believed him.

I think I've figured out how NoA actually works. It doesn't really know much about lanes in absolute terms, like "this section of highway has 4 regular lanes and one entrance ramp acceleration lane which ends in 200ft, and I am currently in the acceleration lane", or "I am currently in lane 2 out of 4" or anything like that. I could be wrong, but based on my experience I think it's much simpler; it more or less knows only "I am in the rightmost lane because there's nothing to my right", or "I am in the leftmost lane because there's nothing to my left". And if it knows it's supposed to be exiting right in 1 mile, it just asks for lane changes to the right until there's nothing to its right.

Extremely disappointing and in my experience -- let me reiterate that this part is not conjecture but is my actual experience from using NoA on urban highways -- it is completely useless for urban highways and for commuting in heavy traffic. Never mind heavy traffic -- I've done two evening commutes that were well past rush hour and traffic was about as light as it ever is in an urban area, and it still couldn't handle it.