Tesla AI Day - 2021

[diplomat33]

I've been following Autonomous driving since before you, and follow the companies pretty closely.

I've been following Tesla since before 2015. In fact, I was following Tesla and supporting their FSD before I even knew about Waymo.

There is no need to ignore anything! The consumer facing products clearly show who's got what.

Not some presentation or demo!

Tesla has released L2 driver assist to the consumer. Waymo has released real L4 FSD to the consumer.

 

[mikes_fsd]

Tesla has released L2 driver assist to the consumer...

- GLOBALLY that statistically saves many more lives than a tiny sandbox deployment of a few dozen cars.

Waymo has released real L4 FSD to the consumer.

In a tiny sandbox that has shadow cars following with techs to help it get unstuck. I know that "Real" I've watched the videos!

 

[mikes_fsd]

I've been following Tesla since before 2015. In fact, I was following Tesla and supporting their FSD before I even knew about Waymo.

My autonomous driving history: Autonomous Car Progress

I started on the other side... I was a Waymo fan way before I knew anything about Tesla -- let alone Autopilot.

 

[JHCCAZ]

Sometimes I wonder if this [Software 1.0 policy/path planning] is a mistake. My understanding is that perception is really the only thing implemented with NNs, and everything else is your typical path finding, FSMs/FuSMs and decision trees.

But we see in real world demos that driving policy requires inference from multiple variables. Like deciding when to pass a double parked car, for instance, requires the consideration of more variables than what (I think) a human engineer could reasonably encode into software. You have to make a guess as to how likely the car in front of you is going to move, and then how likely it is the lane over will be clear.

Most driving is pretty simple, but (as every autonomous car developer has discovered), it's the infrequent excpetions that's holding back true self driving. These exceptions might be too numerous and complicated to be put into policy.

I doubt if you tried to train an NN to understand all typical driving policy, that you would get very far. But I really wonder how well you could get an NN to perform at "best guess" attempts at recovering from situations that don't clearly fall into existing policy. It might perform better than engineers trying to guess, ahead of time, what the unknown unknowns of all driving situations are.

Of course now you have a volatile, unpredictable, difficult to test, black box of code making decisions about life and death. The interesting question is: would that perform better than a human attemtping the impossible task of programming all driving situations into static policy?

First thanks for the thoughtful comment in the midst of the din.

I think the points you're making really touch on the hardest and most uncertain parts of what is still to be solved. This is after you get through the discussion of whether the FSD Beta's current foibles are, or are not, due to any sensor suite inadequacies - not just "can camera vision do it in principle", but "is the Tesla camera suite sufficient". I think that is s unclear but let's move on, back to your topic.

The most compelling demonstration I've seen that justifies optimism, it can be done, is the one by another AK, Alex Kendall of Wayve. You may have seen it but if not I'd urge you to watch it. It shows the car making very intuitive, human-like and confident decisions in London streets where (as he says) the whole concept of a lane is quite fluid. I'm pretty sure this is AI applied to driving path planning and, in any case, is a level of continuity and smoothness that's not apparent in FSD beta (because of hesitancy) nor in Waymo (because of chosen ODD) - at least in the videos that I've seen.

 

[GlmnAlyAirCar]

You don't say!

When @diplomat33 joined this forum he was a Tesla fanboy riding the FSD bandwagon. So, yes, his opinions have changed.

 

[emmz0r]

Whatever helps you sleep at night - I guess!

Seems a lot of the fantasies about FSD have a tendency of becoming reality.
Remember the initial traffic light detection release and the BS the peanut gallery poured on that feature?

Again, you seem to be stuck in the legacy code mentality, which I actually understand, but just because you have little to no imagination of how to get something done, do not ascribe your limitations to those that do and can and are doing something about it!

Can you explain how "2.0" will replace classical code for driving policy? Anything will do.

I have not had that feature fail on me in the life of my 11/2020 build Model Y! Not a single time!

Here it mistook tunnel lane signs for traffic lights and chimed all the damn time so I had to turn it off. So 1000+ fails for me.

The "AI" is dumb as a brick. It only is good for recognition of objects etc. The decision making is all C++ regular code, and will be for quite some time.

Sometimes I wonder if this is a mistake. My understanding is that perception is really the only thing implemented with NNs, and everything else is your typical path finding, FSMs/FuSMs and decision trees.

But we see in real world demos that driving policy requires inference from multiple variables. Like deciding when to pass a double parked car, for instance, requires the consideration of more variables than what (I think) a human engineer could reasonably encode into software. You have to make a guess as to how likely the car in front of you is going to move, and then how likely it is the lane over will be clear.

Most driving is pretty simple, but (as every autonomous car developer has discovered), it's the infrequent excpetions that's holding back true self driving. These exceptions might be too numerous and complicated to be put into policy.

I doubt if you tried to train an NN to understand all typical driving policy, that you would get very far. But I really wonder how well you could get an NN to perform at "best guess" attempts at recovering from situations that don't clearly fall into existing policy. It might perform better than engineers trying to guess, ahead of time, what the unknown unknowns of all driving situations are.

This is what I think as well!

Of course now you have a volatile, unpredictable, difficult to test, black box of code making decisions about life and death. The interesting question is: would that perform better than a human attemtping the impossible task of programming all driving situations into static policy?

You would need to have a NN that remembers things, and sees context in the situation , and train it for that in billions of unique situations.
You can make those things fit into a matrix so it becomes a thing it can process. But not sure how successful it would be. There are many "buts" and "ifs". What happens when 2 scenarios are combined? How does it know which overrides which? Pass the intersection, or drive to the side when the emergency vehicle comes?
ML has no general understanding of things, which is why it's used for instantaneous things that are singular.

 

[Bladerskb]

Many people fail to understand Tesla's mind-blowing FSD accomplishments, but that's ok. Tesla will still deliver despite the naysayers.

lmao, where have you been hiding these past weeks while mobileye has been roaming time square with the first vision only system (supervision), doing things FSD Beta isn't capable of?

 

[mikes_fsd]

Can you explain how "2.0" will replace classical code for driving policy? Anything will do.

Let me ask you this:
Do you believe you can train a NN to understand your surroundings? (The way Tesla claims they have and backed that claim up with FSD Beta releases)

If no, then there is no further discussion that can be had!
If yes, then we can go on to the next part of the discussion!

Assuming you agree that NN can be trained to understand their surroundings (obviously the understanding part is only as good as the dataset that you trained it on and how well it is maintained/labeled).

Take a look at the following scene (src: AIDRIVR

https://www.youtube.com/watch?v=wD_mF0OLJPs

@ 4:03 )

Beside the "normal" stuff like cars lanes, etc the scene gives us a descent idea of what the car understands.
(A) It is 25 mph zone (B) there is a car that is parked down a driveway (C)
Remember that the same NN's are already determining the drivable space as well.

In Beta 8 we saw a little more obvious info in the raw bounding box view with color coding of what each moveable object was doing (driving/walking perpendicular, oncoming, in the same direction etc)

If a NN can be trained to understand its surroundings - like we have seen in the FSD Beta ... it can also be trained to act properly based on the information that it has observed.

Driving policy can be boiled down to 2 actions (really!) Those actions must fit the environment where the car is.
1) acceleration -- or lack thereof -- i.e. brake
2) direction we should go -- i.e. steering

The rest is dependent on your environment.
* slow down for the speed bump vs slam on brakes because a pedestrian stepped into your lane.
* turn the wheel some small degree to follow a slight curve or make a left/right turn
* slow/speed up because speed limit changed.
All of these are dependent on the cars observations.

In this example:
* observe the 25 mph sign - make sure your are in the ~25 mph ballpark
* observe a driveway with/without cars know that there could be stuff coming out from there.
* if the Audi in this scene was switching into our lane observe car and slow down to give room.

If a NN can be trained to understand its environment well, it can be trained to act in that environment as well.
I am pretty certain that that the path planning that we are seeing in the FSD beta is coming as an output from the NN.

I think Dojo is what will be needed to be able to train a NN - not just to perceive the world around it, but also to train it how to behave in that world (i.e. driving policy)

 

[emmz0r]

Let me ask you this:
Do you believe you can train a NN to understand your surroundings? (The way Tesla claims they have and backed that claim up with FSD Beta releases)

If no, then there is no further discussion that can be had!

No.

 

[ohmman]

Moderator Note: A number of posts moved to snippiness. Not everything was snippy, but there was a lot of quoting of previous posts which would have been out of context and orphaned if I'd left them. Regrettably, some included worthwhile contributions to this thread. Apologies for that, and for any orphaned posts that were left in place. I did my best.

Personal attacks are prohibited here and will result in account action. Please keep the discussion on the topic and not the participants.

 

[idriveacar]

No.

I'm curious as to why you think that. I guess it comes down to what you mean by "understanding your surroundings".

Tesla has clearly used this technology to create software that can extract a 3d point cloud from vision. It also seems to consistently classify objects correctly (notable bugs seem mostly to do with traffic lights, but I'm not sure if I've seen those types of bugs in V9 videos). And it can read signs.

After classification, it seems that the software understands a lot about the structure and rules of the road - lanes, speed limit signs, status of lights, etc.

The least mature "understanding" of the world appears to be with the intent of other drivers.

Does the car understand the world in a general way, like humans? No. But depending on your definition of "understand" I think it clearly demonstrates that it is capable, albeit not very well in some aspects.

 

[emmz0r]

I'm curious as to why you think that. I guess it comes down to what you mean by "understanding your surroundings".

Tesla has clearly used this technology to create software that can extract a 3d point cloud from vision. It also seems to consistently classify objects correctly (notable bugs seem mostly to do with traffic lights, but I'm not sure if I've seen those types of bugs in V9 videos). And it can read signs.

After classification, it seems that the software understands a lot about the structure and rules of the road - lanes, speed limit signs, status of lights, etc.

The least mature "understanding" of the world appears to be with the intent of other drivers.

Does the car understand the world in a general way, like humans? No. But depending on your definition of "understand" I think it clearly demonstrates that it is capable, albeit not very well in some aspects.

The reason why I think it's a no go right now, is that these things you are talking about are separate neural nets.
One for detection of this, one for detection of that.
All in a continuous stream of data, reacting to it right there and then with some memory.

There's no "intelligence" in the I - for now. It's only an instant, realtime processing of data based on a trained model. The image stitching is cool, but that also is just a continuous stream of present form data.

In the future they might use DOJO to train it to do everything in all conceivable situations with reward sketching:

Data-Driven Robotics

Large-scale machine learning is one of the greatest triumphs of artificial intelligence in the last decade. The recipe for success typically involves the following ingredients: a large deep network, a massive curated dataset, and a lot of computational power. With this recipe, we witnessed a

sites.google.com

STILL there's no "understanding", just a trained pattern in a black box that "just works"

 

[emmz0r]

Also one thing, they tried to do this sort of ML thing for cut-in detection, not sure how it went. 2 years in the making, I still have to disengage autopilot to let people in from on-ramp merger.

 

[diplomat33]

Dennis Hong teased an image of the Dojo chip on Twitter and hinted that he is involved with Dojo:

https://twitter.com/x/status/1422628009211740160

https://twitter.com/x/status/1422546906341945350

Tesla's in-house Dojo chip teased by legendary engineer ahead of AI Day

Ahead of Tesla’s AI Day scheduled for August 19th, legendary mechanical engineer Dennis Hong has teased a picture of what could be Tesla’s Dojo Chip. While Dojo is a Supercomputer that Tesla Head of Autonomy Andrej Karpathy released photographs of recently, Dojo uses an in-house chip, according...

www.teslarati.com

 

[AndreP]

Waymo and Cruise engineers present at industry conferences and universities all the time.

Don't they care to build hype to unsustainable levels and constantly overpromise but under-deliver?

 

[Daniel in SD]

Don't they care to build hype to unsustainable levels and constantly overpromise but under-deliver?

Well they they haven't exactly delivered on the initial hype either. haha.
Of course they've only taken money from accredited investors who hopefully understood that it was not a "solved problem."

 

[S4WRXTTCS]

Don't they care to build hype to unsustainable levels and constantly overpromise but under-deliver?

I feel like everyone in the autonomous driving industry does that, but to different audiences.

Some hype it to such an extent that if it was any other topic it would be considered fraud.

It's only because it's this unsurmountable problem that everyone is given a pass.

Clearly I made career choice mistake in not entering the autonomous driving industry.

 

[RobDickinson]

That looks an absolute beast of a cpu!

 

[heltok]

That looks an absolute beast of a cpu!

Looks like a beast of an NPU.


Btw, I am not native speaker. It’s an ”en-pee-you” not a right?

 

[JHCCAZ]

Looks like a beast of an NPU.


Btw, I am not native speaker. It’s an ”en-pee-you” not a right?

Correct. You assume the acronym will be spoken as you would verbally sound out its letters, not as the words they represent.