If you don't give a visual or audio response within half second, you're frozen and the user will start hating on you. If you don't show something within 1/10th of a second, you're slow - if you can't actually do something by the time 1/10th of a second passes, you have to at least show a wait cursor. If it's interactive (e.g., dragging or pinch zooming), you can't be more than two ticks (1/30th of a second) behind. Successive refinement at idle time is just about always the right answer.
User events must be treated as real-time interrupts, and all the heavy lifting has to happen at idle time or on a background thread or in another process.
Just stopped into this thread for the first time and SByer has it precisely right. When designing a touch UI, responsiveness is the utmost concern. Very small delays add up to great user frustration. This is why I consider my Android phone "slow" compared to its iPhone contemporaries. Android will periodically stutter and fail to respond.
The reason users start hating on you when your user interface appears to "freeze," as SByer says, is that they don't know whether you processed their input. This is especially likely when dealing with a touchscreen since there is no tactile feedback. So the impatient among us, such as myself, will press again and possibly even three or more times if the pause is more than a second. Once the user interface awakens from its slumber, often the result is a spastic processing of a sequence of "clicks" in rapid fire, which can be even worse than the delay itself.
You'd think that impatient folks like myself would be trained by sluggish devices to just wait. But I can tell you flatly that I never learn. When a device--my Android phone, my Kindle Touch, my AT&T U-Verse DVR, whatever--doesn't respond to something in about half a second, I immediately tap the screen or press the button again. User interfaces absolutely must run at the highest-priority and respond immediately with something, even if it's just an indicator that the input has been received.
I too have been nervously awaiting an opportunity to test the user interface responsiveness of a production-ready Model S. What I have seen on videos to date appears distressingly unresponsive. I can almost feel the tension in the air that exists for those sparing tenths of the second while the Tesla representative has to decide if she should tap the "maximize" button again. You can see that she has been trained to be extremely deliberate with her taps in the videos too. A touch UI needs to work for deliberate, casual,
and rushed use...
Especially a touch UI in a car! Consider the risk of an unresponsive user interface when the user
needs to know that their input was successfully captured immediately because they need to get their eyes back on the road. Knowing myself, I fear the small fits of rage that I will have if the touchscreen doesn't process my rushed input and I have to take my eye off the road for more than a tiny fraction of a second to more deliberately press some on-screen button.