Tesla.com - "Transitioning to Tesla Vision"

[greenmore]

Radar doesn't provide redundancy because radar itself can't drive the car. Multiple cameras provide redundancy.

Redundancy is more related to dealing with hardware failures rather than sensor fusion. Sensor fusion provides more confidence in perceiving environmental objects in most cases. Redundancy is having multiple of the same sensor / system in case one fails.

Redundancy (engineering) - Wikipedia

en.wikipedia.org

None of that is really true.

Multiple cameras in Tesla cars provide a wider field of view than any individual camera instead of redundancy. There's limited overlap between cameras and the way Tesla uses them is also not redundant. They process each camera's image separately and then "stitch" the different data together for an internal bird's eye view representation.

Also not true that redundancy is all about hardware failures. Failures and data artifacts can come from a variety of sources, both internal and environmental. I'm not even sure what to make of the comment about sensor fusion.
Redundancy doesn't mean you have to have multiple of the same sensors.

 

[Huskyf]

This sounds same as with radar.

Yes but I notice in beta fsd in heavy rain or snow or sun in front of camera they doesn't disconnect or alert driver I think the version is better actually with radar if you have ice or snow melt in front of the hood the autopilot is unavailable.

 

[BitJam]

Let me get this straight. They lowered the price, added radar, improved safety features, and bumped the max AP speed over 75 MPH. Excellent! So glad I ordered an MY five days ago.

More seriously, range anxiety has morphed into (no-)radar-anxiety. My order was bungled so I ended up with the cell number of the manager of the nearest Service Center and was told to call if I had any other issues. I was tempted to call with my radar-anxiety but decided not to. I'll let others be the squeaky wheel. I'm very disappointed with how Tesla has handled this so far. It seems obvious this was a business decision due to a parts shortage. Putting lipstick on it only makes it worse. OTOH, I'd be very surprised if this was not fully resolved in some way by the time my car is ready for delivery.

OTOOH, I doubt they will be able to see the car ahead of the car ahead of you or resolve seeing in inclement weather without radar. This is far from ideal. It's never good to get less than what you thought you had paid for. But realistically I doubt these limitations will affect me personally (feel free to use this as my epitaph if I happen to be wrong).

IIRC, when airlines started it was all effectively first class. Flying was very expensive which was a strong filter. Now first class travel and first class airport lounges are prohibitively expensive for most people which keeps out the unwashed hoi polloi. I was excited about getting a Tesla because it felt like I was getting into the first class club. This was bound to change especially if the model Y starts outselling all vehicles, both gas and electric. I wonder if this puzzling radar decision has started the change. You're only flying first class with the models S and X. Owners of the 3 and Y in North America will be traveling coach.

 

[Huskyf]

It does matter though; Radar against a car in front of you helps guide your car to follow that car.

I really do not understand the defense of this move when it degrades features and thus is clearly not ready. Again, why is more sensors ever a bad thing?

Picking up my nerf'd car tomorrow

Mobile Eyes follow Tesla with pure vision

 

[greenmore]

I think almost all fsd developers only use one sensing modality (whichever sensor provides the most accurate information) to measure things like position, velocity, etc.

For example, with Tesla's radar enabled, Tesla is going to rely on the radar for all velocity measurements (for front of the car). But you're right, perhaps if the radar fails, then it may fall back on the camera.

I'm pretty sure most serious FSD developers use multiple sensing modalities. For example you can't read traffic signs or lane markings with Lidar. Lidar is also pretty useless to measure velocity when driving on a featureless straight road. In fact, GPS, IMUs, and mechanical readings like wheel rotations and steering inputs are heavily used for odometry.

 

[YieldFarmer]

And they'll do this in 3 months probably, 6 months definitely.

Fyi, a ranger can't install a radar, it requires calibration that can't be done in the field.

Calibration can be done in the field (aka at home).

1) Install radar (use a level to ensure its flat)
2) Plug in diagnostic ethernet
3) Access diagnostic interface
4) Click calibrate radar (Need an ODIN token for this from Tesla Toolbox. Only body shops and some helpful people around here have them)
5)Drive around for a few miles and it will calibrate

The harder part would be updating your cars actual configuration. Tesla seems to be able to do this remotely, but even the service techs dont have access to this. Perhaps something they could do over the phone is really necessecary.

 

[greenmore]

I have no doubt the algorithms are doing that already. In typical dynamic sensing systems once an object is identified a track is initiated and future positions and velocities are predicted. Even if the object is temporarily obscured from view the system knows it's there, at least for a while or until that hypothesis is disproven. Kinematic history plays a very important role in this track, i.e., where it was seen last, which direction it was headed, and how fast it was going. So it does sort of intuit it you might say.

While I agree with most of this, I'd be careful with using the word algorithm. In the world of neural networks there aren't many algorithms anymore. Ever increasing proportion of the vision code is done by extremely complex neural networks that are black boxes. No one really knows what's happening inside them. They are just trained to provide certain outputs in response to certain inputs, and they do that "somehow". No one really cares how, as long as they work.

 

[Huskyf]

Yes but I notice in beta fsd in heavy rain or snow or sun in front of camera they doesn't disconnect or alert driver I think the beta version is better actually with radar on normal fsd/autopilot if you have ice or snow melt in front of the hood the autopilot is unavailable.

 

[greenmore]

This guy AP'ed in the rain just fine, except for oncoming lane issue, which happens with radar cars as well. I wonder what the other guy's follow distance was, as his problem wasn't the rain, it was the water mist from lead cars?:

https://twitter.com/x/status/1398332256498896897

I agree, this has nothing to do with radars.

 

[greenmore]

Kalman filters are used when you have more than one source of data, but they have errors. It weights which solution is likely correct.
There are not multiple sources of data in a Tesla (especially now). Just vision. One neural net. Nothing to weight.
Neural Nets are actually not good at looking across multiple frames yet. This is a very new area of the science. They look at one image and classify it. Then the next. Then the next.
You can't just go and filter either- any filter causes a delay. Delays you don't want when the camera suddenly sees an obstacle that wasn't there before, or when the light changes state. Filters work great on continuous data that is bounded by a physics model (like the trajectory of a vehicle), but not great on unbounded data.
This might be surprising to you, but it is where the state of the art is right now, and it's why things on Tesla screens can change frame by frame.

This is mostly true, but there indeed are NN based solutions for processing temporal data. And Tesla is actually using them. It doesn't mean they are processing actual video data with NNs, though. They aggregate data from still images then convert those to another format of lower dimensionality and feed those to NNs.

 

[rxlawdude]

Fyi, a ranger can't install a radar, it requires calibration that can't be done in the field.

Cite? From reliable source?

 

[YieldFarmer]

Cite? From reliable source?

Its not accurate, I explained above. Have been going through internal manuals and reports from those able to get elevated access.

 

[rjpjnk]

While I agree with most of this, I'd be careful with using the word algorithm. In the world of neural networks there aren't many algorithms anymore. Ever increasing proportion of the vision code is done by extremely complex neural networks that are black boxes. No one really knows what's happening inside them. They are just trained to provide certain outputs in response to certain inputs, and they do that "somehow". No one really cares how, as long as they work.

I know what you mean. The use of neural networks in cognitive applications has become kind of a black art. Most developers I see don't really care how or why the underlying networks work, just that with adequate training they seem to get the right result most of the time. The expertise being built up today is more along the lines of "applications of NN" rather than theory.

I suspect the if Tesla is using neural nets as they say it is probably for an object recognition/classification layer of some sort, not the dynamic part (just a guess of course). The tracking of these objects over time I suspect is still handled with more conventional predictive estimation algorithms, e.g. Kalman and physics.

Obviously there is an element of sensor fusion as well since up until recently inputs from optical, ultrasound, and radar were al used together in some optimized way. I appears the goal is now to modify some of the software and retrain the system to do the best it can without the radar input. I am optimistic but I think it's going to take a while before we get there.

 

[Demonofelru]

Sorry no, I'm referring to this thread where OP who has both says that vision is objectively worse:

https://www.reddit.com/r/teslamotors/comments/no7ahx

I usually don't like linking accounts between sites to myself, but that was me. It is every bit of as bad as I said if not more just because I was kind of afraid I would get obliterated there posting a negative reaction. It did actually get removed as not helpful, but they put it back up. Supposedly it was an auto mod thing.

Honestly it's weird. I see some saying they had the same type of experience as me, but a couple others say it wasn't bad for them. I guess it could be I just got a defective one, or something, but no matter what that won't fix what is honestly my biggest complaint which is the auto high beams things making it nearly unusable at night. At least not while being comfortable with flashing random people.

 

[rjpjnk]

I usually don't like linking accounts between sites to myself, but that was me. It is every bit of as bad as I said if not more just because I was kind of afraid I would get obliterated there posting a negative reaction. It did actually get removed as not helpful, but they put it back up. Supposedly it was an auto mod thing.

Honestly it's weird. I see some saying they had the same type of experience as me, but a couple others say it wasn't bad for them. I guess it could be I just got a defective one, or something, but no matter what that won't fix what is honestly my biggest complaint which is the auto high beams things making it nearly unusable at night. At least not while being comfortable with flashing random people.

Demonofelru, are you saying this was your story about the terrible experience driving at night and in rain? I cannot believe Tesla would expect customers to be ok with a car that flashes its high beams any old time it wants.

 

[orion2001]

I usually don't like linking accounts between sites to myself, but that was me. It is every bit of as bad as I said if not more just because I was kind of afraid I would get obliterated there posting a negative reaction. It did actually get removed as not helpful, but they put it back up. Supposedly it was an auto mod thing.

Honestly it's weird. I see some saying they had the same type of experience as me, but a couple others say it wasn't bad for them. I guess it could be I just got a defective one, or something, but no matter what that won't fix what is honestly my biggest complaint which is the auto high beams things making it nearly unusable at night. At least not while being comfortable with flashing random people.

Thank you so much for writing that post! There is so much noise on both sides with anything Tesla because unfortunately Tesla is polarizing and Musk doesn't help the situation on that front. I found it invaluable to hear about your first-hand experiences, given your experience with the older MY with radar and in similar circumstances. Most of the videos I've seen have been in perfect daytime conditions where I really am not concerned too much about a vision-only system.

It's disappointing to see someone in your position feel very underwhelmed with the MY as it is today. But it has also really helped me make up my mind on where I stand with my current open order. So for that, thank you! I really do hope that they can resolve things soon rather than have this drag on in the terrible way certain things have dragged on for paying customers.

Cheers!

 

[Demonofelru]

Demonofelru, are you saying this was your story about the terrible experience driving at night and in rain? I cannot believe Tesla would expect customers to be ok with a car that flashes its high beams any old time it wants.

Yes, that was my post on Reddit. Yeah, I know others don’t like the auto high beams, and I disabled it on my old one near immediately. The problem is you can’t even use cruise control now without them on for the new non-radar cars. So you have to decide between flashing people from behind with your brights which usually is a kind of road rage thing and driving manually.

It’s a little better if no one is around, but it still feels awkward even when alone them constantly going on and off. It may be an irrational fear, but I’m afraid there is going to be a cop on the side who sees it and pulls me over since it’s kind of suspicious. I don’t think there’s anything explicitly illegal about it, but the fact was if it makes me less comfortable than just driving for whatever reason than there’s no reason not to just drive. The whole point is making it less stressful, not more.

 

[rjpjnk]

Have you tried the car on a good weather night and does it still try to use high beams for AP?

 

[rjpjnk]

Also, can you link to any discussion where others have experienced this? This is the first I’ve heard of the headlights for AP issue. I sure hope you have a defective car and it can be fixed.

 

[Demonofelru]

Have you tried the car on a good weather night and does it still try to use high beams for AP?

I have not yet. Near the end of the ride home it had stopped raining and still required it though. The wording of the warning makes me believe that the requirement has nothing to do with weather so I would be VERY surprised if it allowed them to be turned off at any time. I did just drive to Costco and noticed even during the day it switches auto on as soon as engaging AP. That being said, no I have not explicitly been able to test clear weather at night.
I am almost definitely going to have to make the same trip tomorrow though and it’s supposed to be clear so will report back if I remember how it handles it.