Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Crazy ideas for improving FSD

This site may earn commission on affiliate links.

DanCar

Active Member
Oct 2, 2013
3,165
4,426
SF Bay Area
  1. Have a periscope that is able to come out of the A pillar to see over cars.
  2. Have a periscope come out of the front bumper with cameras on both sides to improve visibility at intersections. Since it is a periscope , can add self cleaning / keep clean feature when retracted.
  3. Thermal camera to improve visibility at night
  4. Test fovian sensor. It will have better color then current sensors and may have better night vision too.
  5. Use depth sensing sensors.
Please share your crazy thoughts for improvement.
 
This quote from Wikipedia article seems to contradict: ... the Foveon X3 has a greater light-gathering ability.
This is not about taking good pictures that is pleasing to the eye. It is about gathering photons. From a first principle's perspective a Foveon type of sensor should be able to gather more photons since it is not blocking any photons using a color array. The color improvement compared to current FSD image sensors is of secondary concern.
 
I've posted many times about camera placement, night vision performance and also (a bit less) about cleaning issues. There are plenty of inexpensive ways Tesla could enhance the camera placement and viewpoint geometry.

But the rest here is prefaced by the observation that, even given my strong opinions, I also respect and am willing to defer to Tesla's engineering decisions - but it would be a lot easier to do so if they would have covered these topics in a little bit more detail at one of their presentations. I I would urge them to address this at the next autonomy day if there is one coming up.

The periscope concept you mentioned - I won't say it's crazy, but I think it's probably overcomplicated, relatively expensive and doesn't contribute to the vision in a consistent full time way. You identified that it's needed at intersections, but actually the designers want to get the most out of every camera position, full-time. Accidents don't just happen at intersections, edge cases are just that, so why build a complex mechanism to tuck away the camera?

My suggestion was corner cameras integrated into headlights and maybe tail lights. I think this is highly advantageous regarding viewpoint geometry for both looking at cross traffic and, in the forward direction, looking past left-turning traffic to see the high-speed oncoming lanes that are dangerously blocked by those waiting cars. I also think the keeping-clean problem it's helped by a) known surface-treatment methods and b) widely known and relatively inexpensive washer modules if needed.

Though it's still not entirely clear what is coming with HW4, it looks like Tesla has continued to ignore these expert suggestions:). However, if I understand the early leaks about forward plus rear cameras in the repeater modules, these could do almost as well in looking to the side (but not quite as well placed to look past roadside obstacles and adjacent cars), and at least an improvement over the central windshield cameras for looking past oncoming left-turning traffic for UPLs (but still somewhat restricted and sensitive to the ego car's pose).

Regarding night vision, I don't think thermal IR sensing is quite as important as some people do (Sandy Munro is a strong believer in FLIR). I do think that near-infrared (optical) night-vision sensitivity is helpful, and could easily be improved with matching-wavelength IR LED emitters in the headlights, and even in the repeaters etc. This is like common and now cheap surveillance cameras. The beam pattern of the IR emitters doesn't have to be dipped. (I see this as a potential alternative to the smart Matrix adaptive headlights, which seem to have really stalled in terms of widespread adoption. It's not clear that the new Highland M3 even has them.)

I do agree with others that Foveon sensors are not as efficient, nor as versatile in the engineering of color channel selectivity. But I think the biggest problem is that they're proprietary and not widely adopted, so they don't meet the ultra-cheap characteristic that Tesla would be looking for.

Regarding depth sensing, I'm very hopeful regarding the forward HD radar, but we don't have much more detail about that. I don't perceive that there's much chance of side-ooking HD radar. Note that even the more economical Lidar, like the first math production units from Luminar, seem to be configured as primarily forward looking sensors, and it does seem that HD radar is probably I'm more sensible and economical alternative in this current design generation.
 
@JHCCAZ - I'm sure you know more about this than I ever will. But from a crazy idea perspective:

1. No periscope. Use drones with omni-directional cameras at intersections. Auto lands on car otherwise. No, not all accidents are at intersections but I'm sure most are.

2. Mandate that every new car made to include a device that transmits position, direction, and speed. Maybe even a flag for emergency vehicles running with lights engaged. Range 1k yards, primarily forward. Signal readable and decipherable by anyone. Cannot be used to issue citations without corroborating infractions. I'd be surprised if such components would exceed $50.

3. Use the Tesla fleet to maintain at least a rough database of street layouts, signal light changes, AND allow for Waze-like crowd-sourcing for things like potholes and construction. Of course the car has to navigate independently but such mapping could avoid most potholes (IMHO), and use this data as the primary "what to expect" source, with Google maps being secondary. Of course, this data is sent to every other Tesla automatically in that area, and available to other manufacturers for both reading/updating on a subscription basis. There are two intersections near me building new traffic lights but they are not yet hung. I get serious PB at both with "slowing for signal", but it never stops or visualizes a traffic light. Must be map data.

4. I'm sure I saw a push that showed the car reading pavement markings but it doesn't seem to work. Add that functionality to #3. These should help with lane positioning significantly.

$.02 off the top.
 
@JHCCAZ - I'm sure you know more about this than I ever will.
Thanks, but really I dont "know", and it's hard to claim otherwise when the problems aren't yet solved. I have technical opinions based on related experience and/or fundamental principles, but that's quite different from having any inside knowledge of Tesla's plans or internal deliberations. As I said up-thread, there are some things that I don't understand about their architecture and camera-placement conclusions, even though I'm in line with their strong push for minimizing hardware cost and complexity. As a product designer myself, I like to ruminate about all these pros and cons, but that doesn't always translate well to the forum - for people expect you to come down clearly on For or Against [whatever topic].
But from a crazy idea perspective:

1. No periscope. Use drones with omni-directional cameras at intersections. Auto lands on car otherwise. No, not all accidents are at intersections but I'm sure most are.
:) it's been discussed, often in jest, and I think everyone here would love it from a coolness perspective. However, I think it would never "get off the ground" due to cost, complexity, reliability and legal issues at several levels.
2. Mandate that every new car made to include a device that transmits position, direction, and speed. Maybe even a flag for emergency vehicles running with lights engaged. Range 1k yards, primarily forward. Signal readable and decipherable by anyone. Cannot be used to issue citations without corroborating infractions. I'd be surprised if such components would exceed $50.
Here I actually agree with you, and it's a technology and infrastructure called vtV2V and V2X. Maybe a decade ago or more, frequencies were set aside and it was expected to be a coming thing. However, it seems to have died out for various reasons. I can tell you that if you bring it up on the forum here, you will likely be treated to a great deal of contempt, ridicule and explanations of why it's impractical, insecure and unnecessary. I have never been convinced by those anti-V2X arguments but they are promoted by some fairly ill-humored people. So watch out.

I think one of the most interesting potential benefits, aside from the obvious traffic safety motivations, is that the system could grow to allow extremely high traffic density with vanishingly small accident and traffic-jam rates. The benefit would be tens of billions of dollars per year per state in savings by obviating the need for most road widening projects, as well as some dreadfully expensive and typically low-usage public transit projects. But again, promote it here at your own social-media peril.
3. Use the Tesla fleet to maintain at least a rough database of street layouts, signal light changes, AND allow for Waze-like crowd-sourcing for things like potholes and construction. Of course the car has to navigate independently but such mapping could avoid most potholes (IMHO), and use this data as the primary "what to expect" source, with Google maps being secondary. Of course, this data is sent to every other Tesla automatically in that area, and available to other manufacturers for both reading/updating on a subscription basis. There are two intersections near me building new traffic lights but they are not yet hung. I get serious PB at both with "slowing for signal", but it never stops or visualizes a traffic light. Must be map data.
This one, or something close to it, is already in the works. See this discussion after the FSD software foundations of it were uncovered by @verygreen (I think @greentheonly on Twitter, which I don't yet follow but I think he posts more there):
Post in thread 'Elon: FSD Beta tweets'
I think we're all going to be hearing more about this, and hopefully starting to see some benefits, over the next year or so.
4. I'm sure I saw a push that showed the car reading pavement markings but it doesn't seem to work. Add that functionality to #3. These should help with lane positioning significantly.

$.02 off the top.
I'm very surprised that there isn't more road-marking and (maybe more universally) sign reading and reaction built into FSD already. Once again, to understand why it's been so delayed and/or why it hasn't been more highly prioritized, you'd have to be inside the Autopilot team. I'm kind of shocked that Road Closed, School zone signs, Do Not Enter / One Way and many other fairly critical signs, potentially new, temporary or otherwise not reliably mapped, are still apparently not registered. FSD is someone better at not running into a Road Closed sign, but it doesn't seem to honor it.
 
  • Like
Reactions: KArnold
The rest of the quote from Wikipedia on Fovean light gathering abilities is:
Although the Foveon X3 has a greater light-gathering ability, the individual layers do not respond as sharply to the respective colors; thus color-indicating information in the sensor's raw data requires an "aggressive" matrix (i.e., the removal of common-mode signals) to produce color data in a standard color space, which can increase color noise in low-light situations.[15]

That citation at the end is from 2007. I'll probably add a Wikipedia edit to the article to counter someone's claim that Fovean sensors are more light sensitive than Bayer type sensors. I read the Wikipedia article and take it with a grain of salt. It reads more like an advertisement.

Modern non-Fovean sensors are extremely light sensitive and good for very low light situations such as astronomy and security. They are especially sensitive to the IR spectrum and require an IR filter to block that component. Fovean sensors are less sensitive to IR because that is the deepest layer of the sensor, meaning the light has to travel through the other color detecting layers first. I have two Fovean (Sigma) cameras and above ISO 100 the images are very noisy.

My crazy ideas for improvement are Vehicle-to-Vehicle communication and smart highways (such as RFID markings).
 
1. if I disengage and choose a different lane every time I go on route, remember that choice, please
2. Like how I voice-train my phone on first use, allow me to drive train FSD to learn my pace for waiting on stop signs, how much to slow down for turns, etc.
3. Let me record a route once, and replay it every day (i go to pickup my kid, there are specific entrance/exits for the school during school dismissal time, Tesla never learns, I have to disengage every time)
 
  • Like
Reactions: DanCar