As a frame rate junky that loves smoothness, this is something I wanted for a long, long time. Tesla finally implemented hardware video decoding on MCU2 with the 2023 holiday update! This has lots of benefits that I'm going to geek out about.
Because of this, all 3 cameras in the backup camera view is much smoother. Previously, the backup camera was by my eye's estimate running at approximately 24fps, and the side cameras are ~15fps. What's worse is that it was dropping frames randomly, so the frame rate was stuttering and inconsistent and inconsistent across the 3 camera views. Now, the videos are clearly much smoother, and running at the full 36fps, which is the max fps AP2 cameras are capable of.
This also affects the dashcam viewer. Dashcam videos play much smoother, seeking is much faster. It's a beauty for a frame rate junky like me.
Technical stuff:
How is this possible? Tesla used to play the video streams from the 3/4 different cameras using the CPU to decode. The Atom CPU in MCU2 isn't fast enough to decode 3 simultaneous streams of videos without dropping frames, which is why the backup camera can be jerky. But there's a dedicated hardware video decoder on the Atom (capable of simultaneously decoding 15 1080p30 videos), and Tesla finally started taking full advantage of the hardware decoder and because of that all camera views can be played at full frame rate and full resolution completely smoothly, freeing up an enormous amount of CPU cycles for other things.
This has other benefits:
Blind spot camera resolution is much better. It takes about 0.5 to 1 second to start playing a video. Because the left/right repeater cameras need to show up immediately when the blinker is on, Tesla was just playing the 2 video streams for the left/right repeaters in the background, regardless if the blinkers are on. This way the video can pop up immediately when the blinkers are on. But this is taxing on the CPU, and it's a constant burden, so Tesla used a much lower resolution and bitrate video for the repeaters in order to reduce the CPU load. But you can clearly see the blocking artifacts in the videos during motion because of the low bitrate. Now, with hardware decoding, there's no performance penalty, so Tesla is directly playing the same high bitrate video stream used for the dashcam on the screen. It's smoother and higher video quality, it's wonderful!
Maps loading is noticeably faster. Because video decoding is offloaded to the hardware decoder, CPU utilization is down dramatically, and therefore there's a lot more CPU cycles available for maps to load noticeably faster than before.
Unfortunately, there is no hardware video decoding for Tesla Theater apps like YouTube and Netflix. Tesla Theater apps are webapps run on browsers, which is Chromium on Tesla. However, Google has stated they do not want to enable hardware acceleration support for Chromium on Linux. Therefore, all video is CPU decoded on the browser.
Pro tip: When playing YouTube, choose 1440p. YouTube sends AV1 videos for 1080p and below, because AV1 is 30-50% more efficient than VP9. But AV1 is much slower to decode than VP9, and therefore MCU2 can't play 1080p AV1 smoothly. However, MCU2 can (almost) decode 1440p VP9, with ~5-10% dropped frames, which isn't too visible, way better than ~30% dropped frames on 1080p AV1. So instead of dropping to 720p, go up to 1440p instead. It's smoother and higher video quality.
"Interior of Model 3" by Leo Nguyen is licensed under CC BY-SA 4.0.
Admin note: Image added for Blog Feed thumbnail
Because of this, all 3 cameras in the backup camera view is much smoother. Previously, the backup camera was by my eye's estimate running at approximately 24fps, and the side cameras are ~15fps. What's worse is that it was dropping frames randomly, so the frame rate was stuttering and inconsistent and inconsistent across the 3 camera views. Now, the videos are clearly much smoother, and running at the full 36fps, which is the max fps AP2 cameras are capable of.
This also affects the dashcam viewer. Dashcam videos play much smoother, seeking is much faster. It's a beauty for a frame rate junky like me.
Technical stuff:
How is this possible? Tesla used to play the video streams from the 3/4 different cameras using the CPU to decode. The Atom CPU in MCU2 isn't fast enough to decode 3 simultaneous streams of videos without dropping frames, which is why the backup camera can be jerky. But there's a dedicated hardware video decoder on the Atom (capable of simultaneously decoding 15 1080p30 videos), and Tesla finally started taking full advantage of the hardware decoder and because of that all camera views can be played at full frame rate and full resolution completely smoothly, freeing up an enormous amount of CPU cycles for other things.
This has other benefits:
Blind spot camera resolution is much better. It takes about 0.5 to 1 second to start playing a video. Because the left/right repeater cameras need to show up immediately when the blinker is on, Tesla was just playing the 2 video streams for the left/right repeaters in the background, regardless if the blinkers are on. This way the video can pop up immediately when the blinkers are on. But this is taxing on the CPU, and it's a constant burden, so Tesla used a much lower resolution and bitrate video for the repeaters in order to reduce the CPU load. But you can clearly see the blocking artifacts in the videos during motion because of the low bitrate. Now, with hardware decoding, there's no performance penalty, so Tesla is directly playing the same high bitrate video stream used for the dashcam on the screen. It's smoother and higher video quality, it's wonderful!
Maps loading is noticeably faster. Because video decoding is offloaded to the hardware decoder, CPU utilization is down dramatically, and therefore there's a lot more CPU cycles available for maps to load noticeably faster than before.
Unfortunately, there is no hardware video decoding for Tesla Theater apps like YouTube and Netflix. Tesla Theater apps are webapps run on browsers, which is Chromium on Tesla. However, Google has stated they do not want to enable hardware acceleration support for Chromium on Linux. Therefore, all video is CPU decoded on the browser.
Pro tip: When playing YouTube, choose 1440p. YouTube sends AV1 videos for 1080p and below, because AV1 is 30-50% more efficient than VP9. But AV1 is much slower to decode than VP9, and therefore MCU2 can't play 1080p AV1 smoothly. However, MCU2 can (almost) decode 1440p VP9, with ~5-10% dropped frames, which isn't too visible, way better than ~30% dropped frames on 1080p AV1. So instead of dropping to 720p, go up to 1440p instead. It's smoother and higher video quality.
"Interior of Model 3" by Leo Nguyen is licensed under CC BY-SA 4.0.
Admin note: Image added for Blog Feed thumbnail
Last edited:
