Impressed by Waymo's AI Simulation City

[Bladerskb]

I remember talking about Waymo's simulation city and was laughed at by @orion2001 @ZeApelido
People like @EVNow and @powertoold laugh at the idea that Waymo is years ahead in AI research and implementation.

That's perfectly okay. Everyone is entitled to their beliefs. When when we look at actual AI research and results we see a different story.
For example note that all the inventors of NERF either work at Google/Waymo. Google is responsible for probably ~90%+ of all neural net breakthroughs.

I previously said that "Waymo is building a WaymoZero City Simulator using their SurfelGan 3D map that they relight with 24 hour realistic Time of Day, weather, seasons with vectorNet, various sensor simulation NNs, and smart Imitation and RL learned agents that Drago have talked about. A simulator so realistic, not even a human would be able to differentiate it from a real feed from their cameras. A 1:1 scale and match in both geometry and textures would definitely be a game changer."

Guess whose laughing now?

A full implementation of Simulation 2.0 will give birth to Level 5!
Once you can automatically simulate: World Building, Weather, Actor Intelligence, Dynamic objects and scenes, completely end to end.
Its over.
Game. Set. Match.

Block-NeRF

Scalable Large Scene Neural View Synthesis

waymo.com

 

[TresLA]

A simulator so realistic, not even a human would be able to differentiate it from a real feed from their cameras.

That’s quite the claim. I’m sure they’re working on making it better, but the clips above aren’t exactly reality-fooling.

 

[EVNow]

Guess whose laughing now?

A full implementation of Simulation 2.0 will give birth to Level 5!
Once you can automatically simulate: World Building, Weather, Actor Intelligence, Dynamic objects and scenes, completely end to end.
Its over.

LOL. I've seen better graphics from 3rd tier game studios.

I'm sure Waymo drives at level 5 in their simulation - but not in the road adjacent to their service area in SF.

ps :

Game. Set. Match.

Reminds me of all the clickbait Tesla youtube videos.

 

[ZeApelido]

Still need to know what exactly to input into those simulations, and that's still the problem.

If Waymo already had all the data they needed, they could open up services in many places. Cuz they can just simulate it right?

Except instead Waymo is collecting more data in additional cities. Because they don't they don't have enough of the very sparse, very odd and unpredictable events that would eventually occur if they opened up a service.

Actions speak louder than words.

This has always been my point and you just seem to not get it.

 

[impastu]

Let the guy have his "victory"...he clearly needs it with posts like that.

 

[Bladerskb]

LOL. I've seen better graphics from 3rd tier game studios.

3rd tier game studio? There's no such thing as a third tier game studios.
Its triple-AAA game studio and indie game studios.
Gotta love tesla fans.

This literally replaces the tens of thousands of hours it takes artists and engineers to create a tiny game environment.
While it will take Waymo rides around the city and a push of a button.

A triple A game takes 5+ years (with 500-1,000 employees) and most of that time is spent modeling, texturing and building the environment/world.
That's 5 years for an area not even the size of acouple city blocks and its not even 1x1 scale.
Its riddled with repeating assets, repeating textures.

In contrast the real world contain infinite variations of objects and texture.
Something as simple as road texture and bump is so different.

I'm sure Waymo drives at level 5 in their simulation - but not in the road adjacent to their service area in SF.

With Sim 2.0, something you simply can't grasp.
You will be able to simulate an entire country.
Including the road adjacent to your service area.
How is it that you seem not to be able to understand something so damn simple?

When your simulation is indistinguable to your system compared to the real world.
You can jump into San Jose and see how your system performs in the same road that FSD hit the bollard on.
You don't need to drive to San Jose. If it happened at night, you simply turn the dial and its night.
If it happened in the rain, you simply turn the dial and its rain.
If it happened in direct sunlight, you simply turn the dial and its direct sun light.
If it happened in heavy snow storm, you simply turn the dial and you have heavy snow storm IN SAN FRANSICO!!!!

ps :

Reminds me of all the clickbait Tesla youtube videos.

I'm glad you caught that.

 

[powertoold]

All AV "progress" summed up:

https://twitter.com/x/status/1486215976559398915

 

[EVNow]

3rd tier game studio? There's no such thing as a third tier game studios.

Ofcourse noone will claim to be 3rd tier - they call themselves "indie" like some of those "indie" movies or "indie" labels.

Gotta love tesla fans.

Or for that matter ex-MSFT who know a thing or two about game studios.

 

[Wnuk]

I'll be impressed when it becomes available in a vehicle I can purchase and at an affordable price.

Till then the game, set, and match continue in my eyes.

 

[Krash]

Guess whose laughing now?

People with proper grammar?

On a more serious note, this thread title does not get to stand. Send me a Private Message with a replacement title proposal and I’ll add you to the (re)name that title contest.

(update - thread retitled and contest now over. Thank you to all the attendees)

 

[Krash]

"Person on internet demonstrates…

Disqualification. Send a Private Message.

 

[JulienW]

"Person on internet demonstrates... (moderator edit)

Looks like he may have found one.

(moderator comment - also a technical disqualification for not being a private message but I'll allow it because it is so funny)

 

[Bladerskb]

Still need to know what exactly to input into those simulations, and that's still the problem.

huh? you mean google knew exactly what to input into AlphaGo Zero? Into AlphaStar? into Muzero? into AlphaFold?

If Waymo already had all the data they needed, they could open up services in many places. Cuz they can just simulate it right?

This is such a simple and ridiculous way to look at a problem.
Tesla fans do this so much its absolutely nuts.
Whenever they are talking about others. Its completely binary, its either driverless or its garbage. It either works and is available or its garbage. Whenever they talk about Tesla. Their response to Tesla FSD tying to kill you every acouple mins is: It doesn't matter. its in beta, its improving and learning every day.

Except instead Waymo is collecting more data in additional cities. Because they don't they don't have enough of the very sparse, very odd and unpredictable events that would eventually occur if they opened up a service.

That's actually not true. They are collecting data in additional cities with completely different driving dynamics.
For example they test in Phoenix because its the surburbs and has good road and nice weather, but it also has dust storms and heat so it really challenges their hardware to withstand the elements. They still at the same time have to be able to detect and respond to other road users. Driving in phoenix doesn't mean you can just run over pedestrians, rear end cars or get into head on collisions. There are tens of millions of visitors to phoenix each year, Waymo HAS to detect and respond to ALL of them.

They test in SF because its an urban environment with dense traffic and tons of pedestrians
They test in Florida & Seattle because they have heavy rain.
They test in Detroit & New york because they have heavy snow and different driving style.

They don't test because they are looking for sparse very odd or unpredictable events.
Infact the whole "long tail events" is an actual myth.
Cruise has pointed out that what people believe are long tail events actually happens very often even just in SF.
Like dozens of times a day even for a fleet of 100 cars.

The beauty of simulation is you only need something to happen once and it becomes apart of your simulation corpus.

Actions speak louder than words.

This has always been my point and you just seem to not get it.

Really the i'm going to kill you if you turn me on FSD Beta isn't loud enough?

 

[roblab]

I'll be impressed when it becomes available in a vehicle I can purchase and at an affordable price.

Affordable? What does that mean? Affordable to whom? I felt that my Model S was affordable. So was my Model 3. Listen, you get what you pay for. If you want Toyota amenities, a weak gas engine, and pollution, then you'll pay what the masses pay and won't give a ****. However, some of us want more, and some of us want to lower our carbon footprint, something it seems you aren't too worried about.

If you were truly wanting an electric, you can pick up a used Leaf pretty cheap. If you want FSD, you'll have to pay, and Tesla's working on it.

 

[qdeathstar]

A simulation proved nothing. Every 6 hours computers simulate the weather but past 3 days it rarely verifies.

 

[Wnuk]

Affordable? What does that mean? Affordable to whom? I felt that my Model S was affordable. So was my Model 3. Listen, you get what you pay for. If you want Toyota amenities, a weak gas engine, and pollution, then you'll pay what the masses pay and won't give a ****. However, some of us want more, and some of us want to lower our carbon footprint, something it seems you aren't too worried about.

If you were truly wanting an electric, you can pick up a used Leaf pretty cheap. If you want FSD, you'll have to pay, and Tesla's working on it.

I guess you misunderstood. I already have a 2018 model 3 with FSD thank you very much. I enjoy it for a variety of reasons, the largest of which is FSD and it’s potential. So I ‘m not sure how you wrongly jumped to conclusions about what I am or am not worried about. The internet is such a lovely place.

What Waymo offers is not for consumers who want autonomous driving in our personal vehicles. They do not produce a vehicle but make equipment and technology and adapt it onto vehicles made by other manufacturers. It works but looks quite unappealing. What they are doing now is impressive but as it sits today I can not own/use their technology in my vehicle. This is why I say the game is not over. And before you jump to any other conclusions, I do not believe Tesla will solve FSD with their current array of cameras.

 

[BrownOuttaSpec]

If this is for FSD then why is there so much emphasis on houses, trees, power-line poles, tops of buildings and being able to fly around and get views from the air? Wouldn't the more important details be the road, road edges, lane markings, object detection, traffic and people?

Or is this just the very first step like a painter getting out their blank canvas to begin their work and this just shows that they have a very high quality Museo Linen stretched canvas? Meanwhile Tesla....

 

[Bladerskb]

If this is for FSD then why is there so much emphasis on houses, trees, power-line poles, tops of buildings and being able to fly around and get views from the air? Wouldn't the more important details be the road, road edges, lane markings, object detection, traffic and people?

They are just demonstrating that they are not just moving through the same trajectory that the images were captured in, but that they can have a novel view from any angle.

Or is this just the very first step like a painter getting out their blank canvas to begin their work and this just shows that they have a very high quality Museo Linen stretched canvas? Meanwhile Tesla....

Tesla is doing what other were doing in 2016, 2017, 2018, etc... Back in 2019 on autonomy day, Tesla and elon were laughing about others extensive use of simulation, basically calling it useless, saying its akin to grading your own homework. In 2021 they did a complete 180 and finally doing everything the industry was doing with simulation years ago. They praised simulation through out and everything they said about simulation WAS WHAT OTHER WERE SAYING YEARS AGO.

(Parrael Domain in 2018)

Game engine based simulation is sim 1.0. It relies on environment artists who spend thousands of hours modelling cars, pedestrians, and objects to build a tiny world. Even with all the procedural tools its not scalable.

1) its not scalable
2) it requires lots of artists and thousands of hours
3) its not realistic or a representation of the real world.

Its funny i searched two mins paper and i didn't see anything about any other SDC companies. Its crazy how much hype and popularity can control a narrative.

 

[Bladerskb]

Ofcourse noone will claim to be 3rd tier - they call themselves "indie" like some of those "indie" movies or "indie" labels.


Or for that matter ex-MSFT who know a thing or two about game studios.

Show me any "3rd tier" studio that has been able to create a game with a 1x1 scale replica of the real world?

The large open world games we have, gta, watchdogs, cyberpunk, spiderman, the crew, forza horizon, etc all take 5+ years to develop and all pales in a real world comparison.

Its funny because if i just added in there a direct picture output from waymo forward camera in the exact same street with no cars and you still would tell me how your "3rd tier studios" can create better "graphics".

Tell me what do you gain by making statements like this that are obviously not true?
What do you gain here?

 

[clydeiii]

Ok I'll be the sucker who asks the obvious question. The demo is amazing, but how do we get from that to perfectly simulating the real world? AI has to respond not just to the road, but to other cars, pedestrians, animals, etc. When will they simulate those things?