It's definitely not in your head and is a characteristic of the car. I've driven a few different Y's and every single one of them had this issue. The Model Y has unfortunately a loud boomy cabin. Autotrader and some other reviews mention this as well ie: "on British roads it’s let down by an overly-stiff ride and a boomy, rattly cabin — particularly at low speeds".
Also, threads pop up on reddit about it:
I have both a 3 and a Y, interestingly the 3 makes very similar rumble and booming noises, but with significantly less amplitude. It is so much less on the 3 that I have to be intentionally listening for it, but it's there! Hopefully the Y redesign project Juniper that releases in 2024 will have changes to help mitigate this issue.
As for what to do about it, the suggestions towards the end of the Ear/Pain Pressure mega thread do help reduce the booming to a more acceptable level, but there is no silver bullet.