Tesla.com - "Transitioning to Tesla Vision"

[rjpjnk]

...
In order to teach self driving like we taught AlphaGo, we need a driving simulator that is phenomenal, one that a human could hardly tell is a simulation, down to the variety of cars, people, hedges, weather, etc. This sounds like just as hard a problem as building the thing that can drive in this environment

I understand what you are saying, but we may be thinking too small here. Think back to when someone first proposed that a computer could learn a game by simply knowing the basic rules and just making guided random moves to see what worked best. That was so radically different than previous approaches that involved explicitly programming the machine with the very best of human strategies.

It seems impossibly difficult to us now to build a sufficiently realistic driving simulator that a computer could learn from because it just seems to complex, to capture all the nuances or the driving game, right? Well maybe there is a better way to look at it that doesn't depend on a detailed model?

This sounds a lot like the early naysayers regarding computer chess ever beating a real human expert saying it would require a programming approach that looked ahead exhaustively at every move and every conceivable outcome of the board, and this would require nearly infinite computational power.

That was small thinking. We had to go outside the box and look at the problem differently.

 

[cucubits]

So although it is hard to imagine at present, I think it seems reasonable that eventually a computer like this could learn to be the best player at the game "Driving Cars In Streets" given enough time to learn, and with no more sensors than humans have.

But until that time it sure would be nice to have the radar back so the current system works well!

But elon wants $10k for FSD now. It would be all good if we could "eventually" pay for it when it works...

 

[Microterf]

I understand what you are saying, but we may be thinking too small here. Think back to when someone first proposed that a computer could learn a game by simply knowing the basic rules and just making guided random moves to see what worked best. That was so radically different than previous approaches that involved explicitly programming the machine with the very best of human strategies.

It seems impossibly difficult to us now to build a sufficiently realistic driving simulator that a computer could learn from because it just seems to complex, to capture all the nuances or the driving game, right? Well maybe there is a better way to look at it that doesn't depend on a detailed model?

This sounds a lot like the early naysayers regarding computer chess ever beating a real human expert saying it would require a programming approach that looked ahead exhaustively at every move and every conceivable outcome of the board, and this would require nearly infinite computational power.

That was small thinking. We had to go outside the box and look at the problem differently.

I think this is exactly why Tesla and Comma.ai are focused solely on AI. It makes sense that it's the only way forward. They could be wrong, only time will tell. I know that Elon and George Hotz are both get *sugar* done type of people, so the fact they're on the same page here is telling.

 

[RDoc]

The fundamental issue is that the addition of radar from HW2.0 to HW 2.5 and HW 3.0 did not work over the years. The basic issue of stationary objects in the path of a autopilot controlled Tesla were not recognized and avoided. Hence the crashes with trucks and emergency vehicles.

It seems the issue is the matching of radar data with vision data. Tesla never made that work as expected. So the elimination of radar could make autopilot safer. It could reduce accidents with stationary vehicles, phantom breaking and improve night driving. We will soon find out.

Note that Tesla cameras are not limited the same wavelengths of human vision. If the car thinks it can drive with Autopilot, then it likely can regardless of the driver‘s comfort with the auto high beams.

Of course, the driver needs to be comfortable and capable of driving with the current level of skill that Autopilot has. One has to be ready to immediately ”take the wheel”, when Autopilot bails out.

link to Tesla cmos car cameras

What does the AP see at night?

I use Autopilot over 95% of my driving and I trust AP cameras in the night.

forums.tesla.com

The Tesla software organization may well be incapable of implementing sensor fusion, but how can adding radar possibly make the system less safe? The entire vision hardware system is unchanged. If the new vision only system can reduce accidents compared to the old vision plus radar, I don't see any way the new vision system plus radar wouldn't be better even if the engineers can't take full advantage of it. There's no trade off between loading new software and keeping the radar. The radar subsystem hands the CPU the target locations, velocity and approximate sizes. To do that in a vision system requires a lot of processing so it's not at all obvious how they could have an advantage even in CPU use.

"regardless of the driver‘s comfort with the auto high beams" is quite a stipulation! Not every Tesla driver is completely indifferent to other people on the road, nor are local law enforcement.

 

[powertoold]

High beams not really an issue on newest release 2021.4.18.2

 

[rjpjnk]

I can’t really tell what’s happening in that video. Only saw a couple places where high beams were on I think.

 

[powertoold]

I can’t really tell what’s happening in that video. Only saw a couple places where high beams were on I think.

You can watch it at 0.25x speed for real time. Basically, the high beams only turn on when they're not going to annoy other drivers.

 

[techlogik]

High beams not really an issue on newest release 2021.4.18.2

A quick watch of that video shows that the auto high beam was working perfect. I would say it is nearly as good as my daughter's 2019 VW Jetta Premium auto high beam, or worked very close. VWs is limited to a minimum speed. Why do you need high beam at 20mph? It starts to work like 25-30mph or something if I recall. I don't have another reference to auto high beam with any other vehicle, so that is why I made a reference to that vehicle specifically and implementation. Which is kind of laughable that VW has had this working perfect on a $25k car for many years without a hitch back in 2018....but I digress.

But, when there was enough ambient light from the street lights overhead, or it sensed an oncoming car, it switched off immediately. Never saw 1 time in the 4-5 min that it blinded somebody.

Think the auto high beam issue is resolved with the last update. Think the hysteria about that issue can end now. That is all it was anyway, sensor wasn't working properly along with software to sense light, and go back to normal consistently.

 

[gearchruncher]

That was small thinking. We had to go outside the box and look at the problem differently.

Like I was saying, we have to think about self driving completely differently than chess or AlphaGo. So using them as examples of what can be done is kind of pointless, because the methods used there don't really apply. Well, except Waymo, who uses simulation...

What do you think the "out of the box" thinking is that Tesla has that other companies do not in the area of autonomous cars?

One of the issues that we constantly have here is people assuming things. Very few of us skeptics think L4+ Autonomy is impossible, yet the optimists are often saying that we're just complete naysayers that can't ever see how it will happen. It's clearly physically possible because humans do it. It's very likely humans will figure it out. Like Kaparthy says, we're in an interesting sliver of time.

The issue here is when and if Tesla is actually a leader in this space, and if they should be betting so much of the company on it and charging money for it already as if it's just around the corner. That silver of time could be 10 or 20 or 50 years. There's one thing we know for sure, it wasn't the 2 years Elon has been suggesting for the last 6 years, so there's a reason to be skeptical that any new release is going to be anything but minor incremental progress, and that maybe Tesla hasn't quite found the magic yet required to be the winners in this space.

 

[powertoold]

Yes, alphago is not directly applicable to fsd.

However, it is applicable in the generalizability of NNs. For example, once Deepmind was able to properly model the alphago NN, they created alphazero, which was a generalized master of perfect information games.

In the same way, if Tesla can use vision to master one aspect of real world prediction (like lane lines), then they'll be able to generalize it to predict essentially all visual elements in the real world.

This is why I was so excited when Tesla released the traffic control feature. To me, it was a proof of concept that fsd is possible with vision.

 

[gearchruncher]

To me, it was a proof of concept that fsd is possible with vision.

It is a good step forward. However, reading static (or near static) information visually looks a lot more like AlphaGo than all the things we need to solve with FSD. The state of a stop light is pretty easy to read, and is discrete. A speed limit sign is absolute. Lane lines are controlled and engineered. These are all the things humans engineered to be simple, and based on rules, just like a board game. Yet getting here took years.

L4+ Autonomy needs to avoid a pothole in the road, react to a police officer waving you through a red light that is irrelevant due to construction, deal with ignoring stop signs on the back of trucks, handle changing lanes in dense stop and go traffic, and determine if a pedestrian near a crosswalk wants to cross (stop required) or is just nearby.

I don't see the fact that a camera can identify a human made stop light with only three states and react properly as "proof" that FSD is possible with vision. Of course FSD is possible with vision, it's how humans do it, we didn't need any proof there. The processing of the data is the hard part, not the sensor, and a 3 year old can understand a stop light, but a 3 year old can't come anywhere near to driving a car in the real world because the higher level processing is so much more complex than simple classification. Driving a car is an overwhelming task for most 16 year olds (and they're pretty bad at it), and even for many 40 year olds, but they are not overwhelmed by understanding where to stop when the light is red.

 

[momo3605]

2021.4.18.2 rolling out quickly now and just popped up on my 2018 Model 3 with FSD, anyone know if this will disable radar and transition us old cars to Tesla Vision? Seems like it's the same version new radar-less Model 3/Ys are coming with.

 

[greenmore]

I used to think so, but now I'm not so sure.

The big difference is that humans have a much broader range of recognition capabilities than current digital neural networks, which are trained to do only a few things.

One simple example is recognizing previously unseen obstacles on the road. Tesla trains its networks to recognize about a 100 different objects on images. These networks will be great at finding these objects, but what happens when something shows up that doesn't look like any of those things?

Radarless AP will gladly plow through most obstacles on the road if they are not recognized. Even with radar, AP will drive into pot holes, drive across tree branches or stuff the radar cannot see. These are all the things that the networks were not trained to recognize.

One solution would be, and perhaps they are doing it already or will do it in the future, to recognize roads themselves using semantic segmentation and see if there are "holes" in the recognized road blobs. Those "holes" will represent something that are likely obstacles to be avoided. Problem is that these "holes" often are not real obstacles and most human drivers would just drive through them, but the car would produce phantom braking as a result.

Again, we just got back to the problem of there are too many things to account for and digital networks can't do them all yet.

 

[powertoold]

I don't see the fact that a camera can identify a human made stop light with only three states and react properly as "proof" that FSD is possible with vision.

Seems like you haven't watched the Karpathy talks lol. The difficult part is knowing what lane you're in and whether the traffic light applies to you, where to stop, etc.

 

[Demonofelru]

Unfortunately 2021.4.18.2 does not make the auto high beams any better for my ride last night. Just goes to show how many different cases there are to cover. Good to see it’s much better when there are more cars on the road and none of the super reflective signs.

 

[gearchruncher]

The difficult part is knowing what lane you're in and whether the traffic light applies to you, where to stop, etc.

Exactly. These are highly engineered situations meant to be easy for humans to figure out. Yet we're just getting computers to understand them (and not even well enough that we're willing to have the car go when it turns green for non youtube influencers)

Still very far away from understanding a pedestrian at a crosswalk or a hand signal by a police officer. Like I keep saying, we're on the slow, long, incremental path, not some magic S curve. All good progress. No way someone that paid for FSD in 2016 is going to see their car L4 drive before it's worn out.

 

[Mardak]

also curious to see how weather independence and day/night independence is solved to reach a 5 with cameras only ... radar reaches farther at night and in bad weather than cameras ever will.

I've gotten more radar-related "take over immediately" than camera-related "functionality reduced" where the former is definitely scarier and more dangerous. The radar failure was from having Autopilot active in relatively light snow (and heavy snow it doesn't really make sense to even turn on).

 

[powertoold]

Exactly. These are highly engineered situations meant to be easy for humans to figure out. Yet we're just getting computers to understand them (and not even well enough that we're willing to have the car go when it turns green for non youtube influencers)

Not sure what you're saying, because the same can be said about perfection information games.

Logic here is:

1) Driving infrastructure is built for vision
2) NN masters one aspect of visual prediction (other than Tesla's AP NNs, there are no known visual NN mastery examples, if you know of one, let us know)
3) Because NNs can be generalized, mastering one visual aspect means it can be done for any visual cue
4) FSD can be achieved with vision

I would define real-world visual mastery as identifying 99.9%+ positives, and less than 0.01% false positives (whatever that means).

 

[rjpjnk]

Exactly. These are highly engineered situations meant to be easy for humans to figure out. Yet we're just getting computers to understand them (and not even well enough that we're willing to have the car go when it turns green for non youtube influencers)

Still very far away from understanding a pedestrian at a crosswalk or a hand signal by a police officer. Like I keep saying, we're on the slow, long, incremental path, not some magic S curve. All good progress. No way someone that paid for FSD in 2016 is going to see their car L4 drive before it's worn out.

Yes, a pedestrian in a crosswalk, or a drunk pedestrian in a crosswalk, or a blind pedestrian in a crosswalk, or a pedestrian giving hand signals vs one pointing a gun at me. So many permutations! Impossible it seems.

We are still just beginners at AI assisted technology. Just infants really. But it is coming. I believe that probably in less than 10 years there will be FSD as depicted today. It will still be primitive but I think it will be possible.

 

[gearchruncher]

I believe that probably in less than 10 years there will be FSD as depicted today. It will still be primitive but I think it will be possible.

What is "primitive" L4? What capabilities can this not have that humans have, while still being allowed to drive around with nobody paying any attention at all?

I personally think any guesses at timeframes are silly. AP1 came out in 2014. Elon famously said self driving was a solved problem 6 years ago. Waymo started in Chandler 3 years ago. Completely closed L2 FSD beta has been near a year now. All sorts of companies are exiting this space. What have you seen change in the last few years that makes you believe we're under a decade away to being able to privately own an L4 car that can get you to work while you sit in the back seat?