There are times I don't want the beams on high. The most common case of this is approaching a rural intersection with a car stopped at the cross street. The car doesn't see headlights or taillights, so if the lever is pushed away from the driver, the car's high beams will be on. In order to not blind the other car, I pull the lever towards me to disable the high beams.
I can think of similar cases I don't want high beams on - pulling into someone's driveway; maybe I don't want to light up their entire house or blind someone coming out of the front door to meet me or get in the car.