Improve your home's insulation and leakiness first, second, and third
This. Every $ spent on insulation will save you money for the lifetime of the building, regardless of the price of heating fuel in the future, so worth looking at as an investment, but assuming that heating fuel increases in price it can look a lot more attractive than putting the money in a bank!
We found it very difficult to get a very high level of insulation as a retrofit for our house, so we built a "winter hibernation" extension to Passive House standards; that needs almost no heating (I think its something like 1kW at -10C, that's for a three bedroom extension, so probably the size of an average house), and by lowering the main house temperature have halved our winter heating fuel, overall.
Insulation also helps keep the heat out, in Summer.
pay for an expert home energy audit
We did that too. Thermal Camera check of outside of the house, on a cold night, after the heating has been on "full" for 8 hours
which pointed out weaknesses in insulation and cold-bridging. We also had an air-leakage test (they put a big fan in place of the front door to pressurise the building and used smoke-traces to see where air was leaking into / out of the building). The main house air-leakage is around 3 (normal housing stock in the UK is 10, maybe 5 if it is half-decent, and our Passive House extension is something like 0.3 - that's "air changes per hour" at a specific pressure), so we have made it far better then the norm, but nothing like as good as it would be if we could retro-fit to Passive House standards.
Key aim of Passive House is that a) no air leakage and b) inside temperature of glass etc. is no less than 4C below room temperature. This means you don't get draughts caused by "falling air" because of convection temperature differences, so you feel more comfortable, even if you turn the temperature down a degree or two, and thus that too saves heating fuel.
I was interested in the original question re: using Solar for heating in Winter. Over here we forget how far South the USA is ... London is 51.5N, Edinburgh 56N, and the Southern tip 50N - that's equivalent to just North of the Canadian border up to half way to the New Territories! Of course we have Gulf Stream to keep us mild, but we regard Winter sun as pretty useless and definitely not enough sunshine to do much with PV - our Winter insolation is 90% less than mid summer
, but USA is much further South so presumably some very usable Solar Energy in Winter.
At work it was considered too difficult, and too inflexible (if office partitions moved in future), to retro fit a wet radiator system (most common type of winter heating in UK), so we fitted 100% electric instead using "Far infrared" heaters. These heat you (and furniture etc.) rather than the air, so you feel warm from the moment they are turned on, even if air temperature is cold. They have worked out surprisingly good value (taking into account the capital cost of fitting a new boiler and plumbing all the radiators) and I think, in particular, they would be a good choice in a bedroom - e.g. when you are going to bed / getting up in the morning, and the room is chilly. They will make you feel warm, without having to heat the whole room, for the 10-15 minutes it takes you to get ready for bed / get up in the morning.
My general view on heat-pump, for a wet radiator heating system, is that it only really works if the wet system is low-temperature. Over here that would be an under-floor system around 40C water temperature, whereas our wall radiators are fed at 60C - 80C. For us wall radiators, at 40C, would need to be very large, which would then take up a lot of wall space. Our house has concrete floors, so retro fitting underfloor heating would be a major undertaking .... or shorten all the doors and raise the floor ... and lose ceiling height
