Update: I took it in and the technician confirmed one of the headlights was louder than it should be. He said it was the fan inside the headlights that's making the noise.
They replaced the headlight and now the noise is better. It's still kind of loud compared to how quiet the rest of the car is, but it doesn't bug me anymore.
They replaced the headlight and now the noise is better. It's still kind of loud compared to how quiet the rest of the car is, but it doesn't bug me anymore.