He did NOT say Stop Signs are hard. He said that their is a long tail of special circumstances which can occur along with the detection of a stop sign. That's what the training of the neural net does, and why data volume is important, along with the ability to query the fleet for a specific type of detection.
Further, Andrei said that Tesla creates and stores low resolution maps so that the NAV system knows that there is a stop sign at a particular intersection; the task is to find it visually. This is the way humans drive when they are familar with a route.
Confidence is based on understanding. Tesla's approach is rock solid, and like the Matrix, inevitable, Mr. Anderson.