Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Neural Networks

This site may earn commission on affiliate links.
That would require very reliable metadata down to where in the picture the sign is located. Otherwize the captcha service that needs to be 100% reliable would not be anywhere near. With that kind of metadata, there would be no reason to run this through the captcha service at all, you could just train your system on the data directly.
I'm pretty sure that's what was happening. Common captcha instructions I've seen are to label street signs, signals, cars, buses, storefronts, and so on, and I'm guessing that was to validate how well the system was doing.

Edit - Putting it another way, I could see the captchas I've come across being used for training and for validation.
 
  • Like
Reactions: scaesare
I’m toying with this idea that there are three exponential trends in autonomous vehicle technology. By “exponential” I mean doubling every 1-2 years or faster. The three trends are:

1. Neural network size (based on jimmy_d’s observations)

2. Training data

3. Compute

Larger neural networks can utilize larger training data sets. More compute allows larger neural networks to be trained on more data. So, the three trends are interdependent.

More compute also allows for a) generation of more training data via simulation and b) discovering better neural network designs through neural architecture search and automated hyperparameter optimization.

One might reason that the exponential growth in these three inputs will result in exponential progress on autonomous vehicles.
 
  • Informative
Reactions: aubreymcfato
I’m toying with this idea that there are three exponential trends in autonomous vehicle technology. By “exponential” I mean doubling every 1-2 years or faster. The three trends are:

1. Neural network size (based on jimmy_d’s observations)

2. Training data

3. Compute

Larger neural networks can utilize larger training data sets. More compute allows larger neural networks to be trained on more data. So, the three trends are interdependent.

More compute also allows for a) generation of more training data via simulation and b) discovering better neural network designs through neural architecture search and automated hyperparameter optimization.

I didn't see this link posted in this thread, I think this is very relevant:
Tesla Autopilot Miles | MIT Human-Centered AI
 
  • Informative
Reactions: Off Shore
He claims he's always right. But, the problem with that claim is a cynic is usually right more often than they are not. He also only chooses to go after the most optimistic of the Tesla bunch, and not the more realistic ones.

He can be optimistic about MobileEye because he can always blame someone else if it doesn't do what he claims. He also knows that Neural Networks won't get us there alone. He's not even what I'd consider to be a vision advocate, and I'm not either. So lots of things have to come together on a vehicle to even match what Blader would bet on.

MobileEye isn't really a horse he can bet on as they have no control over what happens when its used in a vehicle. So instead he's just a frustrated individual waiting for everything he's been promised in tech demos. I think he knows the very real fear that we consumers might never get L5. That might end up being fleet vehicles only. That's very much the fear I have.

I put my money on a Tesla not because I think they will get there first, but that it will make for an interesting journey.

There isn't anything like this thread anywhere else. There is nothing in consumers hands like it. I say this as both a good thing, and a bad thing.

Bladar is here because of that. He knows there isn't anywhere else to go. All the action is happening here because this is the wild west.


Notice you have never seen me advocate a car over Tesla for the sole purpose of being a car. I have nothing against Tesla other than the fact that its a breading ground for myth after myth and I usually can't help but call out outrageous myths relating to autonomous driving.

You say i'm a cynic but frankly, i'm just a realist. If i was a cynic my spotless record would at-least have some blemish by now. Yet i continue unscathed.

I called out Nav on autopilot 2 years ago and detailed exactly what its gonna consist and people laughed at me and now its been proven to be nothing more than a fancy gimmick. They don't call me bulls-eye for no reason. Besides, The avalanche of myths will flourish once .41 drops with nav on autopilot.
 
Last edited:
the more interesting aspect of the change to the camera interface is that camera frames are being processed in pairs. These two pairs are likely time-offset by some small delay - 10ms to 100ms I’d guess - allowing each processed camera input to see motion.
Would it be a lot more complicated to process frames in triplets, so that acceleration as well as speed relative to our car can be derived? This could better evaluate risk of collision and reduce false positives.
 
I called out Nav on autopilot 2 years ago and detailed exactly what its gonna consist and people laughed at me and now its been proven to be nothing more than a fancy gimmick. They don't call me bulls-eye for no reason. Besides, The avalanche of myths will flourish once .41 drops with nav on autopilot.

You keep thinking you're the only one that felt like FSD was a complete gimmick. Lots of us including myself knew that the sensors of AP2 would never allow for FSD. There also wasn't a regulatory pathway to it.

I also knew that even parity with AP1 would take a LOT longer than the 1-2 months or whatever it was that Tesla was claiming.

Back then I claimed that Tesla would take at least 6 months for parity. It actually turned out to be a lot longer, and so technically I was wrong. Do I feel bad about being wrong? No, because I was trying to be as optimistic as I could be. That claim of 6 months was an optimistic claim.

I don't know what you claimed because there isn't much point to being right about something like that anyways, and so it's not something I would have paid much attention to.

Why is being right that important to you? Doesn't that limit your ability to put your head out on the table, and your ability to read a threads energy?

This entire thread is Jimmy stating things to the best of his knowledge where he acknowledges there is a lot that he can't know. He accepts that he might be wrong, and he's entirely humble in his delivery. He takes chances, and he admits when he's wrong.

For a thread like this I think it's important to be a bit optimistic, and to think in terms of what could they do considering a best case scenario.

This is a thread for people who are hopeful. Of course it's a breeding ground for myths because that's a bit of where the energy is. If you'd paid attention to most any end-user focused neural network thread they are ALL way more optimistic than the systems can deliver.

To be honest I've never even seen a neural network work correctly even 99.99% of the time let alone more. Except maybe a hot dog, and not a hot dog one.

Despite that I'll try to be optimistic in this kind of thread. Sure I might throw some cold water on the more ridiculous claims, but for the most part I'll just take it all with a grain of salt. It's really not that big of a deal to spend much time with.

I tend to be more optimistic in this kind of thread than I am elsewhere. Like take drive-on-nav for example. I think most people are highly optimistic about that, and I'm in the "Not sure if it's going to work well" camp. I say that because I'm not sure if a neural network will get us there without rear-radar. It's a little hard for me to see a car changing lanes automatically using just a neural network without any kind of rear-radar to back it up. I'm concerned that the edge cases aren't really edge cases at all, and they're hitting limits of what the current sensor system can do.

I certainly don't want to be right, and it's not even something I'll put my vote down on. Instead I'll remain hopeful, and I'll stamp the within 3 months we'll have drive-on-nav with confirmation required and 6 months without confirmation required.

Just out of curiosity where are you putting the drive-on-nav without user confirmation in terms of time-frame? Something that's doable with AP2? Or will it require more? Can it be done with the sensors on the car in a safe manner? Where things like madmax will work?

People tend to get so focused on things a long ways away that we lose focus of the challenge of even the simpler stuff.
 
Would it be a lot more complicated to process frames in triplets, so that acceleration as well as speed relative to our car can be derived? This could better evaluate risk of collision and reduce false positives.


I agree, acceleration (or lack thereof) is useful in predicting problems, if the trend continues. Same with velocity. Car is going to t-bone us unless the velocity drops or deceleration increases. We're safe unless as long as velocity/ acceleration doesn't increase. Trend vs safet margin is a tough problem....

Yes, three frames is more complicated (by over 50%). Are three frames needed?
The next pair of frames will indicate the new speed. If it deals in units, the acceleration can be calculated from the two speed numbers.

Concider frames Z,A,B,C:
Z,A give velocity at time of frame A
A,B give velocity at time of frame B,
B,C give velocity at time of frame C.
From the two velocity numbers it could back out acceleration.
Vs
Z,A,B then A,B,C . More processing, but not necessarily more information.
 
Would it be a lot more complicated to process frames in triplets, so that acceleration as well as speed relative to our car can be derived? This could better evaluate risk of collision and reduce false positives.

Yes, more complicated and requiring a larger network. But if Karpathy is going to do everything in "software V2" rather than just the low-level perception stack, then this will be required in some form, though instead of explicitly feeding in multiple frames at the same time they could, in theory, use a recurrent network (RNN) of some sort to retain memory of previously processed frames at a higher level than the raw pixels. In theory this could work. In practice... we shall see. It's taken them two years to get barely ahead of AP1. Maybe HW3 will give them the power they need to do temporal object tracking entirely in NN but I suspect it will only give them what they need to do the low-level detections reliably enough, and tracking and everything above it in the stack will still be entirely "software 1.0."
 
Yes, more complicated and requiring a larger network. But if Karpathy is going to do everything in "software V2" rather than just the low-level perception stack, then this will be required in some form, though instead of explicitly feeding in multiple frames at the same time they could, in theory, use a recurrent network (RNN) of some sort to retain memory of previously processed frames at a higher level than the raw pixels. In theory this could work. In practice... we shall see. It's taken them two years to get barely ahead of AP1. Maybe HW3 will give them the power they need to do temporal object tracking entirely in NN but I suspect it will only give them what they need to do the low-level detections reliably enough, and tracking and everything above it in the stack will still be entirely "software 1.0."
I assume that whatever reason led AK to process frames in pairs to derive distance and speed rather than use a RNN, would apply as well to deriving acceleration from 3 successive frames.
 
The Autopilot computer has access to data from the speedometer and an inertial measurement unit (IMU). So the car knows its own speed and acceleration without having to estimate it visually.

They are talking about the speed and acceleration of other cars.
AP gets the speed from the wheel sensors (same ones used for ABS, traction, and stability control), which is also where the speedometer gets it from.
 
  • Informative
Reactions: strangecosmos
It would be more succinct to say that successive frames give us relative motion. And since the car knows it's own motion, it can thus integrate the two and pick out stationary objects as well as moving ones.

Yeah, that’s what I was getting at. Also, radar pings apparently give some relative motion information? Or maybe it’s just a binary state: stationary/moving?
 
Last edited:
I certainly don't want to be right, and it's not even something I'll put my vote down on. Instead I'll remain hopeful, and I'll stamp the within 3 months we'll have drive-on-nav with confirmation required and 6 months without confirmation required.

Just out of curiosity where are you putting the drive-on-nav without user confirmation in terms of time-frame? Something that's doable with AP2? Or will it require more? Can it be done with the sensors on the car in a safe manner? Where things like madmax will work?

People tend to get so focused on things a long ways away that we lose focus of the challenge of even the simpler stuff.

I have always believed L3 Highway was possible with AP2 sensor suite and that it is the limit of what they can do with the hardware. They will ofcourse need a new chip. But drive on nav Isn't the L3 highway you are looking for. Most of the perception Tesla needs to drive a L3 highway system isn't even near ready yet. For example detecting road debris, barriers, etc.

Secondly drive on nav is just a simplistic lane change system which has been proven to not be reliable.
@verygreen called it a novel gimmicks. We have it on video making dozens of mistakes and missing easy exits and simply doing dumb stuff.

and you have people with the latest version of it called "Navigate on Autopilot" and they have this to say:
NavOnAP is really dangerous atm. Be safe when testing it!

So if you are looking for a new level in autonomy in the form of L2. This is not it. When it comes you will see it. Elon likes claiming that they could do cross country today if they wanted to by gaming it. But the truth is they can't. As have been proven, their Tesla Vision NN is very much lacking and won't be able to execute a cross country even on a perfect day. Elon isn't the type of guy who would hold back tech that can be deployed. The fact is they have nothing more advanced that could be released as 'Alpha' than what they have in V9.

Again interesting things that @jimmy_d and @verygreen are uncovering but the writing on the wall is driving policy and while everyone is praising Tesla for finally catching up on sensing (though they are not there yet), planning is the Achilles heel in the industry right now.

And like i said.

And about automatic lane change, it will be disappointing because it won't any different than whats happening today. Infact AP1 car could in theory do it.

It won't change lanes in low speed/traffic jams and it won't be doing processing such as... is there obstruction/stopped car/barrier/lane closure upcoming in the lane that i want to turn into?

Basically it wont monitor the environment and will be prone to too many mistakes to be useful

Also as you mentioned freeway transition. when merging into a new freeway and also when changing lane while on the ramp and having to interact with other cars in order to get to the lane to merge into the new freeway. all these things wont be purpose.

its literally a glorified toy that you have to monitor moment by moment. my point is there is nothing ground breaking about that, its pretty basic and will be underwhelming when people realize what it can't do.

This is exactly what happened with @verygreen, disappointment and i said this more than a year ago.
However when advanced planning actually start happening i'm sure they will be able to see it just like they have seen everything else. But no Navigate on Autopilot, won't be a historic moment for computer science as some have tried to claim it will be.

EDIT: This begs the question, when will Tesla actually do cross country? 2019? 2020?
The reason being that most automaker has 2020/2021 circled for some type of L3/L4 highway autopilot.
And a cross country drive consists of 99.9% highway therefore it regulates it to being a highway system.

This means if it happens say in 2020, and Tesla releases an update for an actual L2 Autopilot, it would still mean their billions of miles equated to nothing and didn't give them any advantage as some keep claiming because they would be releasing a less capable system in similar timeframe.
 
Last edited:
  • Like
Reactions: rnortman