Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Technical: How does Autosteer work?

lunitiks

Cool James & Black Teacher
Nov 19, 2016
2,698
5,990
Prawn Island, VC
I'd like to get a deeper understanding of what actually happens from the moment your forward cameras (can) "see" that your lane lines start to curve ahead of you, until your steering wheel physically starts to turn, keeping you centered inside the lane. Both software and hardware-wise.

My problem is that I don’t know jack about the software. OTOH, I think I can contribute with some hardware and wiring information, and from there leave the SW questions for others to answer.

So allow me to begin...
  • Your Daimler/Mercedes steering wheel is attached – through various shafts, rods and intermediate links – to a steering rack that sits between your front drive wheels. Basic theory of operation here.
  • On the steering rack, Tesla has mounted a ZF/Hella Power Steering Motor. It looks like this:
1 - Steering rack.jpg

  • The electronically commutated, brushless Power Steering Motor is what physically forces your car to turn when no hands on the wheel (in Autosteer). The Power Steering Motor works by turning a drive nut via a rubber belt, which - via a series of ball bearings - translates into linear motion of the rack. Linear movement of the rack is transferred to the tie rod ball joints, steering arms and road wheels. Voilà, your car turns.
  • The Power Steering Motor is controlled by a sealed Power Steering ECU, which is mounted right next to the Motor. Looks like this:
2 - Power steering ECU.jpg

  • The Power Steering ECU houses microprocessors and the output stage for the Motor. This thing is (of course) subject to interrogation for diagnostic data and firmware updates by Tesla, which happens through 500 kbd Chassis CANBUS. (BTW, autopilot hardware 2.5 sports a brand new power steering ECU. I don’t have many details about it other than it’s now fused though a new autopilot 2.5 e-fuse. More on that here and here.)
  • The CAN-wires from the Power Steering ECU make very little sense just from looking at the wiring schematics. They are spliced and diced all over the place, with connections to the iBooster (power braking), the ABS / Stability ECU, Air Suspension Controller, OBDII, Instrument Cluster, Gateway / Center Display and CAN Diagnostics. In AP2-vehicles, it also have a link to the AP2ECU).
So what I would really, really like to know is exactly *how* the visual input (to the cameras) translate into steering commands in this Power Steering ECU. What happens in between? What kind of messages does the ECU get – does it get decisions, does it get “suggestions”, does it get “confidence levels”, or something else? Can anyone help me answer this?

@verygreen you’re of course on the top of my list. But anyone, please chime in!

3 - All together.jpg
 

BigD0g

Active Member
Jan 12, 2017
2,019
4,429
Somewhere
AP definitely sends explicit instructions to the gateway, the gateway then confirms the signature on the instructions to make sure they are valid and then the gateway tells the steering to move based on those instructions. The APE tells the software what and where it sees it being lane lines and other cars and obstacles and then the QTCar app picks the algorithm to use.
 
  • Love
Reactions: lunitiks

lunitiks

Cool James & Black Teacher
Nov 19, 2016
2,698
5,990
Prawn Island, VC
AP definitely sends explicit instructions to the gateway, the gateway then confirms the signature on the instructions to make sure they are valid and then the gateway tells the steering to move based on those instructions. The APE tells the software what and where it sees it being lane lines and other cars and obstacles and then the QTCar app picks the algorithm to use.
Anyone got an example (readout) of CANBUS-signals sent to the Power Steering ECU?

(For it to be meaningful, I guess we need to translate it too)
 
Interesting topic here. Here are my thougs:
Maybe for lane detection there isn't a neuronal network. There could be a Hough-Transformation (
).
Next step would calculate your offset from the middle of the line.
How you translate that offset into a steering input? For that you need a control loop. Otherwise you would constantly bounce between your lanes.
So this would be the chain (just an overview. There is plenty other things that AP also does):
  • streaming the video from the cameras with Video for Linux
  • Analyze the Image with Hough-Transformation and neural net.
  • calculate your offset position from the middle of the lane
  • feed the offset into your control loop
  • the control loop calculate the steering angle
 
Last edited:

About Us

Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.

Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


SUPPORT TMC
Top