Tesla.com - "Transitioning to Tesla Vision"

[Daniel in SD]

Waymo fudges their data by driving easier and easier routes. Any routes that increase risk of disengagement or cause problems will be avoided.

JJRicks’ disengagement rate is generously 1 per 500 miles.

I would be surprised if it was even that good. They encourage safety drivers to disengage all the time. In their paper they state that more than 99.9% of disengagements were not necessary to avoid a collision. When operating without a safety driver I imagine they call for remote assistance quite often.
Watching current beta FSD videos I can confidently say that they're nowhere near 1 contact event per 100k miles yet. I suppose Tesla would have to provide the counterfactual simulation results to prove this definitively.

Aren't contact events only very loosely correlated with the route chosen, by their very nature?

Waymo never hit a fixed object or left the road in 6.1 million miles so I would say yes.

 

[diplomat33]

Can't really, no one will be able to verify the data and Tesla won't likely release it, but if someone has V9 in Chandler, we'd be able to assess qualitatively.

In all Waymo videos, we're assessing qualitatively anyway.

So you admit that we can't compare the data quantitatively. At best, we can only make subjective qualitative assessments. So, you cannot make definitive statements about how Tesla's FSD Beta will be better than Waymo.

Waymo fudges their data by driving easier and easier routes. Any routes that increase risk of disengagement or cause problems will be avoided.

JJRicks’ disengagement rate is generously 1 per 500 miles.

Waymo does not fudge their data. The Waymo data was 6M miles over an entire year of driving, from all over the Chandler area, that included the safety driver. It was all routes, not just the "easy routes". You are confusing the general Waymo data with the driverless rides that do sometimes favor easier routes.

EDIT: According to Waymo's paper, only 65k miles out of 6M were driverless. If we assume the driverless rides were the "easy routes", that would mean the "easy routes" were only 1% of the data. When Waymo uses safety drivers, they don't do only "easy routes". So the data was not fudged.

 

[Dan D.]

How do Auto High Beams handle oncoming pedestrians and cyclists who don't reflect or have small lights? High beams still have to be dimmed by law, safety, and courtesy.

 

[AlanSubie4Life]

How do Auto High Beams handle oncoming pedestrians and cyclists who don't reflect or have small lights? High beams still have to be dimmed by law, safety, and courtesy.

Aside from object identification, definitely the system has to have substantial memory (and appropriate path prediction capability) for this case to work correctly. Otherwise it’ll be a wonderful sequence of on/off high beams as the car forgets about the pedestrian or cyclist.

I have no idea how they currently work for this case (if at all).

 

[DGSteig]

Does Tesla Vision (or any other feature for that matter) use the network for processing, or is everything done locally?

 

[Microterf]

Does Tesla Vision (or any other feature for that matter) use the network for processing, or is everything done locally?

Its don't locally. Trained on the servers, but processed locally

 

[johnm]

But elon wants $10k for FSD now. It would be all good if we could "eventually" pay for it when it works...

You can. You don't have to do what "elon wants."

 

[rjpjnk]

Its don't locally. Trained on the servers, but processed locally

Do you have a reference for this?

 

[gearchruncher]

Do you have a reference for this?

The fact that nowhere in the manual does it indicate you need a network connection to use AP.

Plus, logic just tells you that you don't run time sensitive, safety critical systems over public cellular connections, nor could Tesla afford the bandwidth if the car had to send data back and forth while on AP, nor would Tesla have needed to have Hw2, Hw2.5, and HW3 if the processing was done in the cloud.

 

[mikes_fsd]

Do you have a reference for this?

Logic alone should suffice.
But latency is enough to know that no processing is done over then network.

 

[rjpjnk]

Logic alone should suffice.
But latency is enough to know that no processing is done over then network.

The question was about any processing.

Do you think voice recognition runs only locally? How about maps and traffic? I thought these checked in with network processors.

 

[gearchruncher]

You can. You don't have to do what "elon wants."

True for people today that now have 5 years of Elon's history to understand.
For all of those that bought FSD in the past based on Elon's very aggressive marketing, it's very telling that Tesla offers no refunds.

 

[gearchruncher]

Do you think voice recognition runs only locally? How about maps and traffic? I thought these checked in with network processors.

Voice recognition and traffic do use the network, but nobody assumed you were asking about anything but autonomy functions.
Maps are a blend, with much stored locally and/or cached.

 

[rjpjnk]

Voice recognition and traffic do use the network, but nobody assumed you were asking about anything but autonomy functions.
Maps are a blend, with much stored locally and/or cached.

I didn't ask. I was responding to the original question from post 925:

"Does Tesla Vision (or any other feature for that matter) use the network for processing, or is everything done locally?"

 

[mikes_fsd]

The question was about any processing.

Not really, the question was primarily "Tesla Vision" also in a thread about Tesla Vision, but I digress.

Does Tesla Vision (or any other feature for that matter) use the network for processing, or is everything done locally?

Voice recognition works offline as well, when there is no reception it will still turn on the light or turn off the AC.
Obviously, if you ask for it to navigate to Starbucks or play Spotify while there is not connection, you're out of luck.

 

[DGSteig]

Voice recognition works offline as well, when there is no reception it will still turn on the light or turn off the AC.
Obviously, if you ask for it to navigate to Starbucks or play Spotify while there is not connection, you're out of luck.

Thanks for that info, as you say navigation is well documented to use the network. Given that these systems work without the network, how does Tesla get to use them for learning how to improve the system? People complain that they are using the US population of 3/Y owners as a giant Beta test, but wouldn't that require a lot of data be sent?

 

[gearchruncher]

Given that these systems work without the network, how does Tesla get to use them for learning how to improve the system? People complain that they are using the US population of 3/Y owners as a giant Beta test, but wouldn't that require a lot of data be sent?

This is an area of discussion, just how much data Tesla can actually get from cars on the road, and what type of data they get.

We can all agree that this data sharing does not have to happen real time. The cars store the video other data, and upload it afterwards. Tesla waits until your car is on WiFi, as paying for LTE data is very expensive, and your WiFi is free.

The thing is, the car cannot collect all data all the time. Video is 10's of gigabytes per hour, and Tesla can't store or process that much data, and it's clear cars don't upload this much data after every drive (It's your WiFi, you can see how much it sends).

So now Tesla has to program the car locally to determine something is interesting, save that, and upload it later. But this means anything not flagged is lost, and it's an interesting area to discuss how you can detect something that is "interesting" if you simultaneously are trying to learn about that exact thing.

Some people believe Tesla is great at this, and is collecting amazing data from all the billions of miles being driven, and this gives them an amazing advantage in autonomous development, especially with "edge cases". Others say that Tesla is so limited in what they can collect that while worthwhile, it's not going to be the critical path to a generalized solution.

This data collection probably is very useful in the "beta" way- Tesla can tell if you needed to take over for the car, so even just a report of how many disengagements they need tells them something about how well the SW is doing. They get other big data too- maybe all they are using the radar-less 3/Y's for right now is to see how many accidents they get into at 70 MPH before they up the speed limit to 90 MPH.

 

[rjpjnk]

Edit: This is in reply to the comment previously that “logic alone should suffice”, which I disagree with. Not the previous comment.

I am not aware of any published information regarding how Tesla vehicles use or don’t use the network in combination with the car’s computer. Is there some? Logic only suggests very fast real time processing be done locally. Tesla has so much more than that.

My guess would be the car is communicating frequently with Tesla servers and pretty much everything it does is potentially influenced by network processing. I’m sure stuff that has to be fast runs local, but general parameters of even real time processing could be updated by network communication. In any event, the data logs of all sensors and driver interactions could certainly be sent to Tesla if Tesla wanted these logs at any time that is convenient. Perhaps when parked.

 

[KArnold]

It would be interesting to know exactly what Tesla gets. Maybe one of our rooters could chime in. @wk057 ?

I've done a few IT designs in my life. I SPECULATE how I might design such a feed. Even with today's cheap storage I'd want to conserve data RW's along with network bandwidth wherever possible. So it's doubtful Tesla would honestly keep that much of the mundane.

Rather I'd look for events, just that they fired. It's small and easy to send in real time. If I see that, maybe send 30 seconds of camera/telemetry back for further analysis. So when an event like "take over now", or AEB, or override of NAV while on a city street, or ABS use, etc. Of course they could change this event list on the fly via OTA - switching between different specific geographic areas, or even smallish samples. Let's say for a week all stop signs while on NAV for a month to do a sprint.

If anything close I would not be overly concerned about Tesla watching or listening to my conversation, unless during the above event. And even then I actually believe them when they say the videos are not identity-stamped. And I'm paid to be paranoid.

Pure speculation of course. I have no inside info.

 

[mikes_fsd]

Thanks for that info, as you say navigation is well documented to use the network. Given that these systems work without the network, how does Tesla get to use them for learning how to improve the system? People complain that they are using the US population of 3/Y owners as a giant Beta test, but wouldn't that require a lot of data be sent?

Probably worth to go directly to the source: Privacy & Legal | Tesla

specifically, the following sections:

• Autopilot information: To further help develop and improve autonomous safety features, we may collect non-personally identifiable images or short video clips using the car’s external cameras to learn how to recognize things like lane lines, street signs, and traffic light positions. These images and short video clips are not linked to your vehicle identification number and we have safeguards that prevent the search of our internal system for clips associated with a specific car. Separately, if you agree to allow us to collect video clips, Sentry Mode will send recorded video clips linked to your VIN to Tesla for temporary backup. We may also use these clips to help enhance detection for Sentry Mode. You can enable or disable the collection of these clips any time via the “Data Sharing” setting in Controls > Safety & Security.
• Advanced features: We may provide you with advanced features in your Tesla vehicle, such as real-time traffic, Autopilot, and Summon, which make use of the road segment data of your vehicle and we may share this data with partners in a non-personally identifiable manner (e.g., by masking the vehicle identification number) in order to help us provide the service to you. We also may collect similar data in connection with other features, such as the navigation data for the online routing feature, and may share it with business partners in a non-personally identifiable manner, where necessary to provide the feature to you. We also only collect or share this data if you enable this collection, although if you do so, your vehicle may send this data to Tesla and its partners even if you are not actively using a feature that needs this information. You can enable or disable the collection and sharing of this data at any time via the “Data Sharing” setting in Controls > Safety & Security.
• Opting out of data sharing: If you no longer wish us to collect telematics log data or any other data from your Tesla vehicle, please contact us as indicated in the “How to contact us” section below. Please note that if you opt out from the collection of telematics log data or any other data from your Tesla vehicle (with the exception of the Data Sharing setting detailed above), we will not be able to notify you of issues applicable to your vehicle in real time. This may result in your vehicle suffering from reduced functionality, serious damage, or inoperability, and it may also disable many features of your vehicle including periodic software and firmware updates, remote services, and interactivity with mobile applications and in-car features such as location search, Internet radio, voice commands, and web browser functionality.