Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Autonomous Car Progress

This site may earn commission on affiliate links.
Wayve adding language as an input to their E2E model:




Here is a chart that shows the input to the LLM. I wonder if "language instructions" means that you could give a voice command to tell the self-driving car what to do, like "engage autonomous mode" to activate the self-driving or "turn left at next traffic light" of you want the self-driving to take a different route?

DPb9TPS.png

I'm...kinda confused why they have "actions (steering wheel, accelerator, breaks...)" being an input and only a text response.

This feels like they're prodding an LLM to try and explain the action of another network that's it's disconnected from? Is that the goal?
 
I'm...kinda confused why they have "actions (steering wheel, accelerator, breaks...)" being an input and only a text response.

This feels like they're prodding an LLM to try and explain the action of another network that's it's disconnected from? Is that the goal?
90% sure it's mostly for fund raising hype. The LLM:s of today doesn't have the response times required to drive, and you can't fit the GPU-cluster required in the trunk even if you tried...

I doubt pouring driving instructions and legal text into the model for training has a real effect.
 
90% sure it's mostly for fund raising hype. The LLM:s of today doesn't have the response times required to drive, and you can't fit the GPU-cluster required in the trunk even if you tried...

I doubt pouring driving instructions and legal text into the model for training has a real effect.

I don't think there's as much of a computational hurdle as you think there is.

Pure C compilations of task-specific LLMs can run inference on my 3 year old laptop (without a GPU) at about 4 tokens per second. I'm sure that with a decent GPU or NPU, you could run an LLM with sufficient speed to be useful for a driving task.

That being said, I think it's also mostly a gimmick or a shortcut. You can translate some aspects of the driving task into a language domain, get a response from an LLM, and then translate that back into the driving task (like Tesla tried with lane-selection). But it's probably more efficient and accurate in the long-run to build a neural network architecture from scratch that doesn't need to be limited by language.
 
  • Like
Reactions: Doggydogworld
I don't think there's as much of a computational hurdle as you think there is.

Pure C compilations of task-specific LLMs can run inference on my 3 year old laptop (without a GPU) at about 4 tokens per second. I'm sure that with a decent GPU or NPU, you could run an LLM with sufficient speed to be useful for a driving task.

That being said, I think it's also mostly a gimmick or a shortcut. You can translate some aspects of the driving task into a language domain, get a response from an LLM, and then translate that back into the driving task (like Tesla tried with lane-selection). But it's probably more efficient and accurate in the long-run to build a neural network architecture from scratch that doesn't need to be limited by language.
For sub 20ms latency it is.
 
For sub 20ms latency it is.

Still, no where near a "GPU-cluster" that cannot fit in a trunk is required.

Last year, people achieved 19.98 ms latency on a 6 billion parameter LLM running on Intel Xeon CPU via 4-bit quantization: https://neurips2023-enlsp.github.io/papers/paper_21.pdf

And Wayve's models can probably be much smaller than 6 billion parameters if they're narrowly focused on the driving task.
 
Last edited:
  • Informative
Reactions: primedive
Good article advocating for autonomous cars. He makes the case that we should not wait for AVs to be perfect and that AVs can save lives now:

There was a time when I believed that self-driving cars should be held to the standard of airplanes. Every mistake needed to be rigorously understood and any human death was unforgivable. But my view has evolved over time as human drivers have continued to kill tens of thousands of people a year. We need a solution that’s meaningfully better than human drivers, yes, but we shouldn’t wait for perfection before we start getting dangerous human drivers off the streets.

 
Good article advocating for autonomous cars. He makes the case that we should not wait for AVs to be perfect and that AVs can save lives now:




Aside from Waymo, it almost sounds like needing to move the goal posts to remain relevant.
 
Aside from Waymo, it almost sounds like needing to move the goal posts to remain relevant.
The answer is probably somewhere in the middle. It's impossible for there to be zero accidents (or even zero at fault accidents, as per recent examples even with Waymo), so people expecting that are unrealistic. However, on the other hand, people in general won't be satisfied even if the stats say overall they are safer than humans. As soon as the vehicle does something serious that a reasonable human wouldn't do, they would question it.
 
The answer is probably somewhere in the middle. It's impossible for there to be zero accidents (or even zero at fault accidents, as per recent examples even with Waymo), so people expecting that are unrealistic. However, on the other hand, people in general won't be satisfied even if the stats say overall they are safer than humans. As soon as the vehicle does something serious that a reasonable human wouldn't do, they would question it.
How much safer? 10x? 100x?
 
How much safer? 10x? 100x?
Follow the money. Once it hits the right point, insurance companies are going to start asking for data that tells them how many miles you drive - and how much of that is on autonomy. The more autonomy, the lower your rate.

At some point, NHTSA will mandate use of the technology as the next automotive safety feature. Imagine being pulled over by the police for driving manually.
 
Apple threw away $10 billion cold ones on its AV project according to article.

It recently came to light that Apple had canceled its car project, which was internally referred to as ‘Project Titan,’ and with the abandonment, the company has re-assigned the 2,000 employees formerly stationed at the self-driving vehicle division to proceed with work on generative AI.

While some may have found it frustrating that all of those countless hours and energy spent for more than a decade would go up in smoke with the snap of a finger, a new report sheds light that some of Apple’s workforce is actually happy about the autonomous car project coming to an end, even after it is estimated that the technology giant invested around $10 billion trying to compete with Tesla.

Work on ‘Project Titan’ was initialized as a concept, and it commenced shortly after the company was done wrapping up the first Apple Watch in 2014. At the time, the California-based giant had a clear intention, which was to take on Tesla and seize a sizable chunk of the market when it came to electric cars. However, having an ambition is a commendable attribute, but achieving it is a whole different ball game.

As The New York Times reported, some employees were happy to see an end to the car’s development, referring to it as the ‘Titanic disaster.’ It is said that work on ‘Project Titan’ was not severed because of engineering challenges but because it was not financially viable. There were whispers claiming that charging $100,000 per car was insufficient for Apple to enjoy a healthy profit margin, and given that the most affordable Tesla would set you back by $42,990, the company was losing the battle before it even set foot on the EV battlefield. Apple was previously reported to have shifted to Level 2 autonomy down from Level 4, making it completely possible for it to pursue mass production.
 
  • Like
Reactions: mspohr
Apple threw away $10 billion cold ones on its AV project according to article.

It recently came to light that Apple had canceled its car project, which was internally referred to as ‘Project Titan,’ and with the abandonment, the company has re-assigned the 2,000 employees formerly stationed at the self-driving vehicle division to proceed with work on generative AI.

While some may have found it frustrating that all of those countless hours and energy spent for more than a decade would go up in smoke with the snap of a finger, a new report sheds light that some of Apple’s workforce is actually happy about the autonomous car project coming to an end, even after it is estimated that the technology giant invested around $10 billion trying to compete with Tesla.

Work on ‘Project Titan’ was initialized as a concept, and it commenced shortly after the company was done wrapping up the first Apple Watch in 2014. At the time, the California-based giant had a clear intention, which was to take on Tesla and seize a sizable chunk of the market when it came to electric cars. However, having an ambition is a commendable attribute, but achieving it is a whole different ball game.

As The New York Times reported, some employees were happy to see an end to the car’s development, referring to it as the ‘Titanic disaster.’ It is said that work on ‘Project Titan’ was not severed because of engineering challenges but because it was not financially viable. There were whispers claiming that charging $100,000 per car was insufficient for Apple to enjoy a healthy profit margin, and given that the most affordable Tesla would set you back by $42,990, the company was losing the battle before it even set foot on the EV battlefield. Apple was previously reported to have shifted to Level 2 autonomy down from Level 4, making it completely possible for it to pursue mass production.
Considering they can take lots of things into other applications and reallocate high value people resources, I don’t think it’s that big of waste in the long run
 
  • Like
Reactions: DanCar
Apple threw away $10 billion cold ones on its AV project according to article.

It recently came to light that Apple had canceled its car project, which was internally referred to as ‘Project Titan,’ and with the abandonment, the company has re-assigned the 2,000 employees formerly stationed at the self-driving vehicle division to proceed with work on generative AI.

While some may have found it frustrating that all of those countless hours and energy spent for more than a decade would go up in smoke with the snap of a finger, a new report sheds light that some of Apple’s workforce is actually happy about the autonomous car project coming to an end, even after it is estimated that the technology giant invested around $10 billion trying to compete with Tesla.

Work on ‘Project Titan’ was initialized as a concept, and it commenced shortly after the company was done wrapping up the first Apple Watch in 2014. At the time, the California-based giant had a clear intention, which was to take on Tesla and seize a sizable chunk of the market when it came to electric cars. However, having an ambition is a commendable attribute, but achieving it is a whole different ball game.

As The New York Times reported, some employees were happy to see an end to the car’s development, referring to it as the ‘Titanic disaster.’ It is said that work on ‘Project Titan’ was not severed because of engineering challenges but because it was not financially viable. There were whispers claiming that charging $100,000 per car was insufficient for Apple to enjoy a healthy profit margin, and given that the most affordable Tesla would set you back by $42,990, the company was losing the battle before it even set foot on the EV battlefield. Apple was previously reported to have shifted to Level 2 autonomy down from Level 4, making it completely possible for it to pursue mass production.
iPhone costs $10 to manufacture. Can't get that kind of profit on a car.
Best they stick to overpriced shiny things.
 
  • Like
Reactions: DanCar