The driver should already be....looking though right?
I'll give my personal experience in a few situations...
If I am looking forward and I get a PB, I almost immediately override because I am already looking forward and I can see that there isn't a critical issue in front of me....if there is an issue happening down the road ahead of me then there is still time to let back off the accelerator and start going to the brake...if it is really a critical then #1 AEB would have kicked in and me pressing on the accelerator a bit wouldn't do anything anyway, and pressing on the brake(without stomping on it) would only disable the AEB and then the braking force might not be enough... or #2 it wasn't AEB critical yet and when it gets to that point AEB will kick on.
Now if I have my head turned because I am checking blind spot to change lanes or whatever, then I generally go ahead and hit the accelerator a bit to stop any "drastic" slow down, go into more of a coast while at the same time snapping my head and eyes back front to assess the situation...again, if was or turns into a critical issue then AEB should kick in.
But everyone is different, everyone has different driving experiences and capabilities, you can't make everyone happy all of the time and someone will always complain about something. Technology advances and bugs have to get worked out, technology advances and some people hate it no matter what.
can't win.