I live in a high desert climate at 6300 feet. Even now in July the early morning temperatures drop outside to ~ 65F. I thought I would be able to drop my home's temp to close to AM ambient with powered ventilation but I never succeeded. The home air temp would nadir to ~ 72 - 75F and then quickly rise through the rest of the morning, suggesting that the home mass unrelated to radiation had not cooled down to the interior air temperature.
Nowadays I dream of passive ventilation through the night and then use of a heat pump that I run from ~ 6am - 8am off PV. Imagine the COP !!
I have done a lot of experimentation with passive energy transfer, rule of thumb: the most efficient transfer happens with ~15F temperature differential. That last 15F takes a lot longer to achieve.