Gut feel, seat of the pants engineering answer: Folded is worse than not because the car was designed to have the mirrors out at speed. If they're folded, the aero shape is at an angle to the wind.
Easy experiment: Drive 70 mph with the mirrors out. Fold them in. If wind noise gets louder, you're less aerodynamic.
My car's mirrors don't fold electrically, so I just leave them out...