Yes - but for NN to do something even badly takes thousands of examples. Organic NN is very different from CNN. You can read papers on that.
Getting back to the original point, solving one edge case may have nothing to do with other edge cases, infact it may regress the already learnt normal...