Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Tesla Vision and Nvidia - who does what?

This site may earn commission on affiliate links.
So here is the hype for your context

The Tesla Motors and NVIDIA DRIVE Partnership

As per Nvidia's approach lane markings should the last thing in priority EAP should be relying on. With HD mapping , "learn to drive where it's safe" why are we not seeing the benefits of AI with the latest release of AutoSteer on surface streets?

Is Tesla vision not harnessing the power of nvidia AI ? Does Tesla have their own system ?


Right now the approach seems VERY specific and algorithmic rather than an AI based approach. Look for lanes , if you don't find them act strange ! I almost wish I could see the rectangles and color coded objects as the system sees it! Would increase our confidence.

I'm of course hopeful that the future will make our HW2 cars better. What am I missing in the meanwhile?
 
NVIDIA provides the Drive PX 2 GPU hardware, no argument there. Both NVIDIA and Tesla are working on software, that much we know. Where it gets blurry is if there is any sharing. We do know that NVIDIA GPUs are used in Deep Learning all over the place, and they have a low-level library cuDNN that is used by all of the Deep Learning packages. Tesla is likely using one of those packages. NVIDIA has tutorials showing how to use Deep Learning for various tasks. Their own self driving program is using such a package to "learn" driving, and they've demonstrated their car navigating around. Tesla is most likely doing their own training and application of deep learning for AutoPilot. Given that NVIDIA is now partnering with Bosch to market their self driving system, I doubt they are helping Tesla very much (although at this stage in the game, perhaps it would be just as beneficial to everyone to work together towards a greater good, which is lives saved). So on the one hard NVIDIA is a close hardware partner with Tesla, and on the other hand competing with them in software application for self driving.

The way deep learning works is you "train" the weights of a neural network based on lots of input. This is done on a big cluster of machine at Tesla (or maybe in the cloud). Then the weights are sent to the cars, and used in the on-board GPUs to execute the neural network. The in-car network doesn't learn on it's own; however, data (video, sensors, etc.) from events that happen (say, where the driver took over, I am guessing) can be packaged up and sent back to HQ and they can participate in the next round of training. Each training cycle is evaluated against the best one you had from before, and that's why they say the cars just keep getting better.
 
  • Informative
Reactions: DJ 240V and croman
@Dithermaster has it down...

Nvidia don't actually make any complete self-driving systems. What they're doing is building tools and platforms that enable other auto manufacturers to make their own self-driving systems.

In a way, they're doing what they've always done. Nvidia don't make computer games; they make tools, both hardware and software, to enable other companies to make computer games.

You'd be hard pressed to find a game that doesn't have Nvidia tech involved somewhere in the pipeline, whether at design time or render time. Likewise, I think maybe 10 years from now you'll be hard pressed to find any car manufacturer that doesn't use Nvidia tools in the manufacturing or running of their cars. Jensen is a very clever CEO.

Just like in the games industry, Nvidia do also make tech demos to showcase their technology. It's not to showcase it to you and me; it's to showcase it to their partners, to the games companies. I think we've all seen them, though. Sometimes, they're used as GPU benchmarks... usually pretty scenery with rain and particle systems and hard-to-render reflections etc. They're not full games - they're just designed to show off what their software/hardware can do. The same is true for their driving systems; once you sign up as a partner with Nvidia (as Tesla are) you can study their examples, learn from it, even use bits of it if you want... but it's not a full/complete system. It's training/showcase/advertising material that forms a springboard for bespoke development. This is what we've seen with BB8 and the demos of DriveWorks etc...

In the case of self-driving systems, each major auto manufacturer is working on their own system, and some will end up better than others.

It's an interesting counterpoint to Mobileye. What they're doing is a much more complete "in-house" solution; you buy the silicon and the software and plug it in to your car. I imagine Mobileye will dominate the 'reasonably priced car' market, and will do very well with smaller car companies that can't afford the engineering teams required to build their own systems. I'd be amazed if their relationship with Audi, Mercedes etc lasts - eventually these companies will want to differentiate... and waiting for Mobileye to innovate and release will become problematic for them (this is no doubt one of the reasons Elon said their split was 'inevitable')

Nvidia also make Mapworks - these are tools that enable mapping companies to make HD maps. Here, TomTom etc etc... these mapping companies are all using Nvidia tech to build their maps. Let them specialise in creating really great maps... they're good at it!

Basically, Nvidia looked at what's necessary to get to full self-driving cars, and then set about making the toolchain that others will need to make it a reality.

AP2 in its current state is not the same as the self-driving system; there will be some overlap, but it's missing the mapping ingredient. Both vision/detection and maps are important, but they have a symbiotic relationship. You can only get so far with vision, and only so far with maps. Together, you can go really, really far.

It's unlikely you'll get complete FSD for the entire world for a long time yet. However, major cities, major highways etc - the mapping companies are already making and refining those HD map areas, because it makes sense to start with high density and easy to cover areas first. Those will be the areas where the FSD is enabled. Where there's no map, Tesla cars will probably fall back on same techniques that AP use today (driver assist).

I would imagine that a Tesla Vision car could drive from LA to New York today, if it had the map. I suppose Musk believes they'll have that ready by the end of the year...

Just like computer games; there are amazing ones and average ones. We're hoping that Tesla are the Rockstar of the self-driving world - but it'll come down to their own talent and ingenuity to leverage the tools and platforms that Nvidia (and others) provide, and successfully augment with their own tech. Tesla want their own system, so that they can update it whenever, accelerate the timeline, push with unique features... plus, Musk has a propensity to make and own as much as possible in house - something he learned from SpaceX (pretty clear from reading his biography).
 
Last edited:
  • Informative
Reactions: Falkirk and DJ 240V