From my experience with high power servo motors - low latency is not a big deal, but it's dangerous. Accidental slam on power and you lose control of half megawatt motors connected to the ground.
High acceleration is relatively 'dangerous' full stop (0-60 in 3.2s for P3D). But if you're gonna go that route, then you may as well make the experience as immersive and fun as possible, and sub-50ms latency I think would help achieve that and give a much more natural feel. At least Tesla should have low latency as an option for those who know the risks and are confident in their ability to use the accel pedal safely.
I suspect technical difficulties (e.g: implementing Obstacle aware acceleration, general software lag, and physical restrictions) are more likely to be responsible for the apx. 100-200ms latency we're seeing, rather than safety, though I may be wrong, and you may be right about Tesla artificially adding this lag.
FWIW, the Jaguar i-Pace has a terrible half second lag! Must feel pretty unnatural.