It's quite possible that lidar-quality environment data can be reliably and consistently be constructed from pure vision. The problem is that it takes hundreds of milliseconds to do so, whereas with lidar you have the 3D data instantly. That's a huge latency difference, and is why Tesla is constantly working to shave off milliseconds here and there. Perhaps HW4 or HW5 will cut this to tens of milliseconds, which would close the lidar gap considerably. I generally agree with Tesla's overall approach, but I think Elon's timeline ("safer than a human by the end of the year") is hopelessly optimistic. (Oxymoron notwithstanding.)
The main thing wrong with Tesla's approach is they sold something incapable of doing what was promised. Not just what was originally promised, but Elon has consistently over the years oversold the promise of what the hardware could do.
This has resulted in a ton of owners being invested in either what we have now or what we feel is realistic for Tesla to upgrade on our cars.
The most popular of those (in my own head) is the idea that the HW4 computer plus upgraded vision sensors will somehow give the car the hardware sensor suite necessary to accomplish full self driving. It will certainly improve it, but it likely won't fulfill the original promise. I don't question that Tesla can close the gap with faster computer plus better vision sensors, but there will still be the gap inherent in how the two technologies work.
The sensor world is also constantly evolving with new technologies or approaches, but were stuck. We're anchored to a 2016 promise, and HW that doesn't meet the needs of AP let alone L4 FSD. Our cars love telling us how the sensors are blinded or that it needs to reduce speed because of a little rain.
We're so stuck in the old HW that there are lots of us with 2018 vehicles who aren't taking advantage of the current market situation to get a newer car for not that much more money (relatively speaking). I haven't because what I have now changes on a fairly constant basis. That would be really enjoyable if the manufacture had good customer engagement to make sure those were positive changes, but they don't so a lot of those things are negative. Things like the autolights no longer turning on in the rain during the day like they used to.
If I'm really being honest with myself the only reason I still have my Tesla is due to the pandemic. From a surviving the pandemic point of view Tesla's approach was spectacular and the sales numbers show this. They were also able to delay introduction of new HW without much consequence, and they had additional time to work on FSD while their customer base was stuck.
In 2023 I wouldn't be the least bit surprised if Tesla changed the Sensor suite with HW4 to such a degree that it made HW3 obsolete, and unable to upgrade. Where they already know now the wall they're going to hit with HW3. It will be the right call, but it will piss off a lot of existing customers.