FSD v12.x (end to end AI)

[Bitdepth]

Yes, this I can see. I can also envision a bunch of cars frozen and blocking the way because the emergency vehicle can't be seen. Not a trivial problem.

US Patent for Detecting and responding to sirens Patent (Patent # 11,854,390 issued December 26, 2023) - Justia Patents Search

The technology relates to detecting and responding to emergency vehicles. This may include using a plurality of microphones to detect a siren noise corresponding to an emergency vehicle and to estimate a bearing of the emergency vehicle. This estimated bearing is compared to map information to...

patents.justia.com

 

[spacecoin]

What propaganda? You have not viewed the clip yet!!

I haven't? Quick summary: Musk is a genius, Lidar is a fools errand. Teslas is the best. No one knows why Teslas aren't full autonomous yet...

 

[stopcrazypp]

I haven't? Quick summary: Musk is a genius, Lidar is a fools errand. Teslas is the best. No one knows why Teslas aren't full autonomous yet...

Not at all a fair summary. For example, just the lidar part is a huge mischaracterization. They mentioned Elon said lidar was a fools errand, but he goes into detail about the history of why lidar was so prominent in self driving cars and also its advantages.

Basically it gained prominence from DARPA's self driving challenge. Basically before NNs were really a thing, it was a relatively easy way to localize the car and also do obstacle detection, so relatively easy to get a demo car working. With the growth of NNs, localization can be done via cameras only, so that advantage has faded. He also says although they work great for small scale demos, using lidar for localization and maintaining high precision maps is not going to be scalable, so its use for this purpose will continue to fade as vision improves.

A sanity check shows the Chinese makes (Huawei and XPeng) seem to have come to the same conclusions (they are abandoning trying to build their own high precision maps and instead are investing on how to improve the systems to a point that they can drive using conventional navigation maps):

https://www.reddit.com/r/teslainvestorsclub/comments/12yyuqr

As for its continual use (the host asked that given lidar costs have gone down, does it make sense to add it back as an additional sensor) he says lidar is "extremely useful" for object detection and determining the speed/direction of vehicles, but that he was personally pessimistic about its continual use in cars. He predicts at least for the assistive driving application, even though a lot of the closest competitors to Tesla in the space (which he points out as Chinese makes, Ford/GM he says is far behind) have thrown in a lidar in their vehicle, their next iterations will remove it.

A quick search as a sanity check shows this to be the case. The XPeng P5, which was the first production car to feature a lidar sensor for ADAS, have dropped it in the 2024 model year:
Xpeng launches new P5: Removes LiDAR option, cuts to just 2 trims

I haven't watched the whole thing yet, but he goes over a lot on how AP evolved over the years, especially in the use of NN. Basically for AP their first in house efforts basically had minimal NNs and had used hard coding even for parts of lane detection. At the time, there were only a few research papers on the use of NN for cars and not a whole lot sources to reference on using it, so they had to built a lot of it themselves from scratch. Andrej was really the one who pushed much more use of NN. Personally I remember from the tech presentations, Andrej was basically using the most cutting edge methods (that was just in academia) and applying it to AP. It's fairly informative, even if you ignore all the investor talk. Of course Tesla haters will just ignore all the informative parts and only focus on the opinions that they disagree with.

 

[spacecoin]

A sanity check shows the Chinese makes (Huawei and XPeng) seem to have come to the same conclusions (they are abandoning trying to build their own high precision maps and instead are investing on how to improve the systems to a point that they can drive using conventional navigation maps):

https://www.reddit.com/r/teslainvestorsclub/comments/12yyuqr

That makes sense if you settle for L2 and super limited L3.

He predicts at least for the assistive driving application, even though a lot of the closest competitors to Tesla in the space (which he points out as Chinese makes, Ford/GM he says is far behind) have thrown in a lidar in their vehicle, their next iterations will remove it.

No one ever claimed you need more than cameras for ADAS and everyone is chasing costs at this point in time. Lidar will be standard on most cars at some point if the price keeps dropping at this rate.

A quick search as a sanity check shows this to be the case. The XPeng P5, which was the first production car to feature a lidar sensor for ADAS, have dropped it in the 2024 model year:
Xpeng launches new P5: Removes LiDAR option, cuts to just 2 trims

Same thing.

I haven't watched the whole thing yet, but he goes over a lot on how AP evolved over the years, especially in the use of NN. Basically for AP their first in house efforts basically had minimal NNs and had used hard coding even for parts of lane detection. At the time, there were only a few research papers on the use of NN for cars and not a whole lot sources to reference on using it, so they had to built a lot of it themselves from scratch. Andrej was really the one who pushed much more use of NN. Personally I remember from the tech presentations, Andrej was basically using the most cutting edge methods (that was just in academia) and applying it to AP. It's fairly informative, even if you ignore all the investor talk. Of course Tesla haters will just ignore all the informative parts and only focus on the opinions that they disagree with.

Nothing AK did at Tesla was "cutting edge" compared to what the rest of the autonomy industry was at the time. It can be summed up as:

1) Move to more NN:s
2) Copy whatever everyone else (in CV, Waymo) is doing and tout it as a novel approach. Again, Nothing put into AP was "cutting edge" (ie first mover) nor a Tesla "invention".

If one doesn't know anything else outside of Tesla, it's easy to get impressed by all the buzzwords flying around. Camera only will not be enough for full autonomy this decade.

 

[daktari]

Excellent comparison of V11 driven by AI Driver and V12 by Omar. Video is narrated by AI Driver. V12 certainly did a much better job than V11 in fairly heavy vehicle and pedestrian traffic. If you cannot watch the entire video start around 8:30. Definitely makes me look forward even more to trying V12.

It looks more fluid but still have disengagments and near crashes on short trip. The most interesting is to see how *sugar* v11 really was, even if the hype was so big.

I guess it is an influencer trick, push down the previous version to make the new excel. In reality,
same but different.

So far away from non-driver ride.

 

[willow_hiller]

It looks more fluid but still have disengagments and near crashes on short trip.

What "near crashes" are you talking about? The one V12 disengagement was due to it incorrectly assuming another car stopped at a stop sign was not moving, and pulling around it.

At no point did it seem like V12 was going to crash into another vehicle. Maybe if he hadn't disengaged, the other car would have hit him out of road-rage, but that's about it.

 

[PianoAl]

I'm good at predicting the future: V12 will initially avoid potholes really well but NHTSA will ask Tesla to recall FSD to fix lane drifting without due caution

And then "every Tesla ever made will be RECALLED" so that the font for the pothole warning is made larger.

 

[daktari]

What "near crashes" are you talking about? The one V12 disengagement was due to it incorrectly assuming another car stopped at a stop sign was not moving, and pulling around it.

At no point did it seem like V12 was going to crash into another vehicle. Maybe if he hadn't disengaged, the other car would have hit him out of road-rage, but that's about it.

Watch AI drivers video from the start. Almost crashes into some cones and barriers

 

[kabin]

What "near crashes" are you talking about? The one V12 disengagement was due to it incorrectly assuming another car stopped at a stop sign was not moving, and pulling around it.

At no point did it seem like V12 was going to crash into another vehicle. Maybe if he hadn't disengaged, the other car would have hit him out of road-rage, but that's about it.

V12 hastily and incorrectly assessed the lead vehicle to be stationary. V12 went around it and then cut in front of the moving vehicle. I'd call that a near crash as two cars wanted to be at the same location at the same time. Some are calling v12 more responsive and assertive but at times it appears to be old fashion poor execution.

Here's another near crash.

And another near crash.

 

[stopcrazypp]

That makes sense if you settle for L2 and super limited L3.

No one ever claimed you need more than cameras for ADAS and everyone is chasing costs at this point in time. Lidar will be standard on most cars at some point if the price keeps dropping at this rate.


Same thing.

But it's already dropping and have dropped to much less expensive than even a few years ago and the opposite is happening.

Nothing AK did at Tesla was "cutting edge" compared to what the rest of the autonomy industry was at the time. It can be summed up as:

1) Move to more NN:s
2) Copy whatever everyone else (in CV, Waymo) is doing and tout it as a novel approach. Again, Nothing put into AP was "cutting edge" (ie first mover) nor a Tesla "invention".

If one doesn't know anything else outside of Tesla, it's easy to get impressed by all the buzzwords flying around. Camera only will not be enough for full autonomy this decade.

It's cutting edge for applying it to a mass market car that is in hundreds of thousands of volume (and now millions). Cutting edge doesn't mean they are necessarily the first to use the method or the only one, but rather the short duration between when a method is released and its actual use in a commerical product. For example in late 2021, Tesla was using Regnets, which just came out a year before.

 

[mgs333]

Still no one aside from Tesla's chief marketing officer has gotten v12? And other YouTubers are reduced to doing videos along with Omar or commenting on his videos?

This sends a really powerful message to all the YouTubers that sucking at Tesla's teat the hardest will get you preferential treatment and bigger YouTube checks. I wonder if at least some of them will start to be more like Omar in their videos in the future.

 

[Bladerskb]

Not at all a fair summary. For example, just the lidar part is a huge mischaracterization. They mentioned Elon said lidar was a fools errand, but he goes into detail about the history of why lidar was so prominent in self driving cars and also its advantages.

It still very prominent in self driving. There is no L4 self driving car that doesn't use Lidar. Its advantages still remains.

Basically it gained prominence from DARPA's self driving challenge. Basically before NNs were really a thing, it was a relatively easy way to localize the car and also do obstacle detection, so relatively easy to get a demo car working. With the growth of NNs, localization can be done via cameras only, so that advantage has faded.

No it hasn't, the advantage is still there.

He also says although they work great for small scale demos, using lidar for localization and maintaining high precision maps is not going to be scalable, so its use for this purpose will continue to fade as vision improves.

Another point he was wrong in, among all the other points.
There are L4 self driving cars in multiple cities, working in multiple road types (suburb, city, urban & highway) and multiple weather conditions (sunny, light rain, moderate to heavy rain, light fog, moderate to heavy fog) 24/7.

So beyond "small scale demos". Secondly even Andrej Karpathy no longer spews the "not scalable" narrative.

"but the approach itself is fairly general and scalable"

Self-driving as a case study for AGI

Musings of a Computer Scientist.

web.archive.org

Lidar hasn't faded. Lidar literally went from ~0 units shipped when that statement were made to ~million units shipped.

A sanity check shows the Chinese makes (Huawei and XPeng) seem to have come to the same conclusions (they are abandoning trying to build their own high precision maps and instead are investing on how to improve the systems to a point that they can drive using conventional navigation maps):

https://www.reddit.com/r/teslainvestorsclub/comments/12yyuqr

They abandoned it due to their lack of software engineering capability to develop crowdsourced HD mapping that would be required for consumer cars with a wide ODD. It has nothing to do with lidar as HD map. HD mapping isn't constrained to just lidar, there's vision based HD map, radar based HD map, etc. You can create HD map from the output of your vision NN.

As for its continual use (the host asked that given lidar costs have gone down, does it make sense to add it back as an additional sensor) he says lidar is "extremely useful" for object detection and determining the speed/direction of vehicles, but that he was personally pessimistic about its continual use in cars. He predicts at least for the assistive driving application, even though a lot of the closest competitors to Tesla in the space (which he points out as Chinese makes, Ford/GM he says is far behind) have thrown in a lidar in their vehicle,

Yes the cost of lidar has gone down multiple orders of magnitude (from $70k) to a few hundred dollars.
And the quality of lidar resolution output has gone up multiple orders of magnitude.
Most people want to mention that computer vision using NN has gotten better but they never mention that #1, Lidar resolution and range has also gotten tremendously better, #2 the improvement in NN also helps Lidar.

their next iterations will remove it. quick search as a sanity check shows this to be the case. The XPeng P5, which was the first production car to feature a lidar sensor for ADAS, have dropped it in the 2024 model year:
Xpeng launches new P5: Removes LiDAR option, cuts to just 2 trims

Yeah you would think so if you were coming from an already established conclusion and walking your way backwards.
But if you actually look into the details you would know that first, its wasn't the first car with lidar.
Second, this is simply just a recategorization of a model. Its not that Xpeng is going away from Lidar.
By removing Lidar from the P5, they also removed City NPG and Lidar LCC, which was what the lidar was used for.

So no they are NOT removing it. As I have pointed out, Lidar shipment went from ~0 to ~million units.
And Its only increasing. If companies were going away from lidar, lidar shipment should be decreasing rather than increasing 'exponentially'.
The whole argument is frankly non-sensical.

I haven't watched the whole thing yet, but he goes over a lot on how AP evolved over the years, especially in the use of NN. Basically for AP their first in house efforts basically had minimal NNs and had used hard coding even for parts of lane detection. At the time, there were only a few research papers on the use of NN for cars and not a whole lot sources to reference on using it, so they had to built a lot of it themselves from scratch. Andrej was really the one who pushed much more use of NN. Personally I remember from the tech presentations, Andrej was basically using the most cutting edge methods (that was just in academia) and applying it to AP. It's fairly informative, even if you ignore all the investor talk. Of course Tesla haters will just ignore all the informative parts and only focus on the opinions that they disagree with.

This couldn't be any further from the truth. Its indisputable proof that Tesla were multi-year behind what was in academia and what was being used by self driving companies. This emphasizes why people shouldn't get their information from a single bubble while ignoring everyone else. I can't tell you how many times i have pointed Tesla fans to several tech presentations of several SDC companies and have them all refuse to watch, telling me they only need to watch Tesla. This exactly is the result of it.

To a man born and raised in a cave, the cave is all he knows and if you met him in his cave. He will exuberantly want to tell you about the 7 wonders of his cave.

 

[willow_hiller]

Watch AI drivers video from the start. Almost crashes into some cones and barriers

First of all, those were presented in the context of regressions, not the documented drive. We don't know how many drives it took to collect those examples of regressions, so your assertion that they happened during a "short trip" is unfounded. During the entire V11 vs V12 comparison drive, there were no "near crashes."

And second, nobody says they "crashed" into a cone. You're exaggerating the severity of the regressive.

 

[AlanSubie4Life]

Here's another near crash.

"Occupancy network is gone, man!"

These guys, really believing everything that they hear.

Though to be fair the car behaved as if it were gone.

 

[KLam]

It still very prominent in self driving. There is no L4 self driving car that doesn't use Lidar. Its advantages still remains.

No it hasn't, the advantage is still there.

Another point he was wrong in, among all the other points.
There are L4 self driving cars in multiple cities, working in multiple road types (suburb, city, urban & highway) and multiple weather conditions (sunny, light rain, moderate to heavy rain, light fog, moderate to heavy fog) 24/7.

So beyond "small scale demos". Secondly even Andrej Karpathy no longer spews the "not scalable" narrative.

"but the approach itself is fairly general and scalable"

Self-driving as a case study for AGI

Musings of a Computer Scientist.

web.archive.org

Lidar hasn't faded. Lidar literally went from ~0 units shipped when that statement were made to ~million units shipped.

They abandoned it due to their lack of software engineering capability to develop crowdsourced HD mapping that would be required for consumer cars with a wide ODD. It has nothing to do with lidar as HD map. HD mapping isn't constrained to just lidar, there's vision based HD map, radar based HD map, etc. You can create HD map from the output of your vision NN.

Yes the cost of lidar has gone down multiple orders of magnitude (from $70k) to a few hundred dollars.
And the quality of lidar resolution output has gone up multiple orders of magnitude.
Most people want to mention that computer vision using NN has gotten better but they never mention that #1, Lidar resolution and range has also gotten tremendously better, #2 the improvement in NN also helps Lidar.

Yeah you would think so if you were coming from an already established conclusion and walking your way backwards.
But if you actually look into the details you would know that first, its wasn't the first car with lidar.
Second, this is simply just a recategorization of a model. Its not that Xpeng is going away from Lidar.
By removing Lidar from the P5, they also removed City NPG and Lidar LCC, which was what the lidar was used for.

So no they are NOT removing it. As I have pointed out, Lidar shipment went from ~0 to ~million units.
And Its only increasing. If companies were going away from lidar, lidar shipment should be decreasing rather than increasing 'exponentially'.
The whole argument is frankly non-sensical.

This couldn't be any further from the truth. Its indisputable proof that Tesla were multi-year behind what was in academia and what was being used by self driving companies. This emphasizes why people shouldn't get their information from a single bubble while ignoring everyone else. I can't tell you how many times i have pointed Tesla fans to several tech presentations of several SDC companies and have them all refuse to watch, telling me they only need to watch Tesla. This exactly is the result of it.

To a man born and raised in a cave, the cave is all he knows and if you met him in his cave. He will exuberantly want to tell you about the 7 wonders of his cave.

So, where are those supposed to be way ahead developments? I am waiting for a replacement of my Teslas.

I seriously doubt the 7 years ex-Tesla engineer is/was living in a cave! I know he used to live in Silicon Valley and now in Texas.

 

[kabin]

So, where are those supposed to be way ahead developments? I am waiting for a replacement of my Teslas.

I seriously doubt the 7 years ex-Tesla engineer is/was living in a cave! I know he used to live in Silicon Valley and now in Texas.

I think the point was having a hammer so everything looks like a nail.

 

[SOULPEDL]

I thought maybe I'd learn something over here. All I got was that people complain about FSD and it's a group that I don't recognize. I did appreciate some of the hyperbole because the car did try to hit a road marker/sign. Ya, oops.

By personally seeing it grow since 2018 on multiple Tesla's, I'm not surprised to see these little things on such a major release - V12 appears way better than what I currently have. Past "new" version quirks were 10x as bad and it did 1/10 of what it does today. To me that's a 10x10 or 100 fold increase. This is not hyperbolic, and it's likely way more than 10x functionality considering people here warned me not to use it on city streets, but I did anyway, "look... no lines!" I collected the videos, they'll be fun to bring out someday. Bottom line V12 is a kitten compare to some of my experiences. You have no idea.

I have a theory on why V12 is not being released broadly this one time. First, I have no doubt that most could handle these grey areas just fine. I would love to get it, just for the smoothness alone. So I wonder if it just boils down to not needing more data, indicated by how some things locally are still not corrected after all these years. So the hiring was happening for FSD drivers at Tesla, one came up in Tempe that I saw. I happen to think that's plenty to test the system in or around each major city.

SNL is on... maybe I'll get back to this post later.

___________ later that day... Edit: I'm back... Elon on drugs in FUD Times Square may have triggered the idea for an SNL skit, lol.

So why don't we have FSD 12 yet?

I believe they're getting ready for amazing Autonomy roll out to sync up with a new use-case implementation. Something like when you whistle into the App, the car will come to you. It has to be believable and awesome with real-world utility. Tesla want's us to all talk about it (and defend against the biggest FUD wave ever anticipated) while some new utility is rolled out. Why all the FUD? Pure economics and politics both.

So far, the more Tesla claims it's better, the more people see all the... fires if you know what I mean. So they are limiting the ability for errors to become pure FUD stories by keeping it under wraps. Danny O' is getting desperate, pressure is building, Mercedes what?

Prediction: The V12 rollout is delayed because they have all the data they need for now (said earlier, must be true). The constraint is only data crunching using feedback from Tesla Employee Drivers. We are not needed at this time, FUD is minimized, and it won't be long because the improvement could save lives, and a larger sample size will be useful to prove it. The ability to pull over and stop is huge. Reverse is likely in the works.

One yard line folks. "Let's Go!"

 

[TK211X]

They need to hurry up and push V12. Everything around me is wet at this point.

 

[Mullermn]

I thought maybe I'd learn something over here. All I got was that people complain about FSD and it's a group that I don't recognize.

A lot of posters on this forum have Tesla branded kidney stones

 

[SOULPEDL]

A lot of posters on this forum have Tesla branded kidney stones

Like you have no idea.

---> Edit... just to keep my train of thought.

Yes, and in the past FSD was worse.

My current version isn't exactly great either, even though much improved. But in the past, FSD releases had fewer Beta Testers to screw up and go bitch, and frankly it wasn't worthy enough for the average driver, nor the media et al to be of a serious threat. But it sure feels like it's getting that way now, just look at this thread.

As the population of existing FSD users widens, we are likely at the level of public use and awareness that is causing Tesla to be more covert in V12 release. I think they realize they need to manage perception here - the path of least resistance. Scrutiny is peaking. Which leads me to conclude a wider release is iterating much like past upgrades, just internally to Tesla this time using their own drivers plus more of the archives (likely plentiful).

The most amazing part of V12 that can I see is steering smoothness and intersection staging/creep. Yes, those quirks on our versions are nerve racking!!! But I haven't seen turns into a pylon in a long, long time, so it's a bit early in learning when I compare to prior releases.

Publicly (including Wall St), FSD is moving from being judged as just a rich kid's toy into an extremely valuable utility about to scale up. So ANY and ALL quirks will be on all the front pages with V12, (along with a dozen other car commercials appearing to do the same thing of course). Soon, Joe Plumber will think FSD is about to drive off a bridge just like in the movies. It's an easy fear trigger burned into our brains. This thread is going to get even hotter yet.

Not sure I'm gonna stick around for an all out mud fight (unless it's something really funny), but to quote BTO, I bet "You Ain't Seen Nothin' Yet".