This is refreshing to see mostly correct answers on an electrical installation thread for once

.
You should be able to tell rather easily if you can use the existing wire to pull in copper instead. Give it a tug on one end and see how easily it moves. If it was actually installed to code, it won't. Romex requires staples near junction boxes/panels, plus strain reliefs when going in and out of boxes (that part can be overcome but it's sometimes a pain). So likely the answer is no, you will need to open the drywall.
If you leave the Aluminum wire, yes you need the anti-oxidizing goo. Also, you will need to occasionally (1-2x per year) go check and re-tighten all terminal screws. Aluminum softens over time. I have had to do this with an outlet used at a family member's house (it overheated once rather significantly, tipping me off to this).
Yes, you need a receptacle that is rated for Al wire if you keep that. If it isn't, you can make a copper jumper and splice it (recommend Polaris connectors if you want some something small, or an outboard 60A non-fused disconnect (like those used for A/C units) if you want something cheap). The same is true if you put in an HPWC -- Gen 3 units are rated for copper only.
Make sure you use a GFCI breaker if your locale uses NEC 2017 or newer (most do). Not needed for HPWC. 40A is a fine choice for UMC usage. Agree that 60A is not going to be a good choice on a 125A service unless you do a load calc.
Finally, get it permitted and inspected per your AHJ.