Tesla's BEVnet that predicts roadway geometry from vision is essentially a global HD map in a compressed format. At least when it's fully trained and labeled from global roads. Instead of storing explicit, human-understandable maps, they are encoded as weights in a network, and the maps are unpacked out of the network when you input camera frames instead of GPS coordinates. This has the benefit of working entirely offline, doesn't require precise localization (or any localization), and it doesn't totally fall apart when it encounters an unmapped roadway or stale map data.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
Autonomous Car Progress
- Thread starter stopcrazypp
- Start date
-
- Tags
- Autonomous Vehicles
Andrej Karpathy discussed this a bit in the AI day presentation, mostly as a future state of affairs. It's an understandable concept and kind of exciting to think that the car will be familiar with all roads as long as a few Tesla drivers have been there before.
So I'm fully on board with the concept and looking forward to its realization. However, so far I think there is little evidence that oft-driven routes are making their way into the NN, encoded in the gigantic matrices of weighting coefficients. Certainly we see the FSD beta testers encountering the same route, intersection and lane confusions, release after release, with some improvements but also with fresh pathologies. Even the much-discussed California-rich training advantage is not clearly evident regarding routes and lanes - it comes probably more from a better understanding of California style road design and markings.
At the other end of the spectrum are the North Carolina test videos put out by Rocco Speranza. Compared to California, NC is a nearly Tesla-free zone and there are many unfortunate errors of planning, routing and understanding. For a while he's been chalking those up largely to navigation-map problems and I think that's right.
One other note - when it does happen, I think it still is not within the definition of "HD Maps". Those are specifically centimeter-resolution databases and include an enormous amount of labeled objects, road paintings and everything that doesn't normally move. When the NN-memory map does arrive, it it won't be a so-called HD map, but a huge collection of "I've been here" memory engrams (a great word I learned as a kid by watching the original Star Trek series, long before artificial brains were anything close to achievable).
Here's a very relevant, entertaining and classic song from that same era, this cover by the great Johnny Cash (done earlier by Hank Snow, and there's a great UK version). This is how Tesla will hopefully be in a few years:
Last edited:
Yep, this was mentioned on AI day, but it's a very abstract idea. It's basically a "map" in the sense of a AI that have driven enough in a variety of roads, that just from looking at a particular road feature, it predicts what is the road structure. But it's definitely not anywhere the same as a map as we typically understand it.
But I guess the difference in this approach is that even though currently the NN's understanding of roads is heavily CA Bay Area biased, even when you drop it into an unfamiliar road, there is some basic competence (for example someone in Ukraine who hacked FSD Beta to activate it).
powertoold
Active Member
Hmm, I may be wrong about this, but I'm pretty sure that the road geometry NN was trained on diverse USA roads, so the road geometry NN itself isn't biased to CA Bay Area.
What is biased to CA Bay Area is the driving policy and planning. Basically, preception = USA wide, planning and policy = overfit to Bay Area since that's where most of the engineers are driving their cars. Just my interpretation about what's going on.
Elon's wording is deceptive because "overfit" is often used in relation to NN training to mean that the outputs aren't good at generalizing outside the dataset. But in this case, I think he just means the procedural code that is used for policy and planning is biased towards the Bay Area.
Last edited:
I'm just referring to the fact from the CA DMV filings a majority of testers are in CA and in other discussions there was evidence present in how an example of an offset left turn (meaning at a left turn, the lanes across the street is a bit off) perception error seems to match CA Bay Area road structure, but is different from other areas.
EVNow
Well-Known Member
There might be some CA bias in NN given most of the cars are in CA - but most likely the planner is the one biased to Bar Area.
powertoold
Active Member
True, but when Tesla is collecting road geometry and lane data, they're looking for diversity, so the data will come from all over the USA. I doubt they're asking the fleet for mostly CA data, but I can see what you mean.
Stale, Un-secure, In-accurate and Unscalable.
This is why Mobileye's REM doesn't rely on, nor is it built on any other mapping system. Its completely standalone.
The debate is already over. The only one still trying to talk about it are people demonstrating purposeful ignorance.
This is has been debunk a billion times and still perpetuated by the Tesla fanbase. Which demonstrates again the inability to think logically and use reason when talking about anything Tesla related. The same people talking about NN detecting objects with confidence metrics simply can't wrap their head around how there could be a tent in the middle of the road. Which AV system can accurately pinpoint with cm accuracy where a mismatch in its map is and that the system would then rely on the NN output based on its confidence level.
This is literally AV Myth #5 : "Myth: An AV can only use its sensors, or its map. It cannot use both at the same time. If the sensor input disagrees with the map, it is an irreconcilable problem and the AV can’t work anymore."
Here look at a Waymo clearly crashing into a tent killing everyone inside after a mis-math between its HD map and its sensor...
Again the only one still spreading this myth are people demonstrating willful ignorance .
Last edited:
Good article
As Electric Car Makers Ante Up Billions, Software Is Ace in the Hole
spectrum.ieee.org/electric-cars
As Electric Car Makers Ante Up Billions, Software Is Ace in the Hole
spectrum.ieee.org/electric-cars
Tesla's FSD obviously does not need HD maps of most roads, as it drives almost everywhere just happily and safely without them. If at all, it might be improved by HD maps of difficult junctions.
Has the thought been mentioned or discussed that HD maps might not be needed everywhere, but only for certain difficult places?
And even there a full HD map might not be needed. Perhaps some formalized hints could do the trick already.
Has the thought been mentioned or discussed that HD maps might not be needed everywhere, but only for certain difficult places?
And even there a full HD map might not be needed. Perhaps some formalized hints could do the trick already.
The TSLAQ crowd is stuck in looney land and still does mind-numbing mental gymnastics when it comes to Tesla. Tomorrow (10/20) is earnings. Tesla had record deliveries this quarter and I'm sure many TSLAQ disciples are praying that Tesla will finally declare bankwuptcy.
#1 TSLAQ fact: Tesla loses $$$ money on every car they build. They keep building and selling more and more cars every quarter. Therefore, larger losses = bankwuptcy.
Tesla is delivering cars, collecting data, updating their NNs, deploying updating firmware, ...etc... Tesla has the scaling and profit parts figured out. Now they're working on the icing on the cake.
I've been using AP with my Y for <1.5 years and it's been great on freeways and surface streets in the Bay Area. Note that I navigate myself and only allow the car to follow the road. I watch Bjorn's videos who tests virtually everything out there. Tesla's AP/FSD is probably the best or near the top.
I'm a bit surprised that Tesla is doing so well. Tesla is working on so many things and are neck-on-neck with the pure plays like Mobileye and Waymo. I don't know about "Chinese Tech" other than many are using American technology. Things can change in a hurry, but I don't think Tesla will be standing still. Notice how all of these various companies compares themselves to Tesla?
#1 TSLAQ fact: Tesla loses $$$ money on every car they build. They keep building and selling more and more cars every quarter. Therefore, larger losses = bankwuptcy.
Tesla is delivering cars, collecting data, updating their NNs, deploying updating firmware, ...etc... Tesla has the scaling and profit parts figured out. Now they're working on the icing on the cake.
Moving goal posts / adjusting to reality. Atleast they are delivering cars.
INFINITI LE Concept – A Vision of Zero Emission Luxury Revealed at New York International Auto Show
The INFINITI LE Concept, which makes its world debut at the 2012 New York International Auto Show, is more than just the latest in the series of dramatically styled, technologically advanced concepts from INFINITI.usa.infinitinews.com
"In a rapidly changing world, INFINITI is well ahead of the game."
I've been using AP with my Y for <1.5 years and it's been great on freeways and surface streets in the Bay Area. Note that I navigate myself and only allow the car to follow the road. I watch Bjorn's videos who tests virtually everything out there. Tesla's AP/FSD is probably the best or near the top.
I'm a bit surprised that Tesla is doing so well. Tesla is working on so many things and are neck-on-neck with the pure plays like Mobileye and Waymo. I don't know about "Chinese Tech" other than many are using American technology. Things can change in a hurry, but I don't think Tesla will be standing still. Notice how all of these various companies compares themselves to Tesla?
It does look like it got kinda stuck there. Do you have the full video for this clip? I'm not doubting your point BTW, I'm just curious about this clip in particular.
Yesterday Elon joined a webmeeting with the top 200 VW managers, VW will copy the Tesla approach and will in-source as much as possible. Already 5k employees of Cariad are working on the IT platform for VW, that number will be 10k in 2025. Tesla is miles ahead to competitors, no doubt. The main question is if the current approach and vision will be the right one for the long term and L3+ …… will L4/5 happen in the next 10 years or only in local robot taxis in big cities? In that case Tesla can improve the current FSD feature until L3, now at L2.2 ;-)
powertoold
Active Member
If I was Tesla I'd just be L2 forever and proud of it. Why would anyone want to be L4/5, now you're liable for accidents, what business would want that? Nah just be L2 forever and focus on solving the hard AI problems of self-driving while selling intermediate deliverables to customers along the way.
rxlawdude
Active Member
I think consumers would be perfectly happy for liability to shift to the manufacturer of L4/5 systems. The manufacturers, of course, not so much.
Consumers would still need collision, comprehensive and other insurance outside of the liability coverage, but that would save a significant chunk of consumers' auto insurance.
I can foresee a time where Level 2 will be changed to driver assist only, in other words you have to actually be steering all the time. The car can nudge you or beep for hazards. The car can control speed.
Anything where the car controls steering might have to be Level 3, in other words the liability shifts to the manufacturer.
Let's say we have millions of Level 2 cars soon from most manufacturers. Most systems are mostly capable of not crashing, except when they're not. Almost everyone is sort-of paying attention. Of course some others are drunk, sleeping, and in the backseat.
If we have bad Level 2 systems that people are abusing it won't be tolerated for long. I can imagine they'll be told to make better systems, make them Level 3 and no more hands-free Level 2.
Just an opinion.
EVNow
Well-Known Member
By who ? Will never happen in US, atleast. Regulators are always nice to manufacturers - they know which side of the bread is buttered.
Take a look at Ms. Cummings twitter feed to get a sense of how she thinks of Tesla (and other AV companies). Not sure if it belongs in this thread, but an interesting choice.
Similar threads
- Article
