They use kerosene (refined lamp oil) for the Falcon rockets and methane for the mars rocket (Starship). Neither is any worse than all the diesel or gasoline that's being used everywhere. They do use a small amount of hypergolic fuel on the dragon spacecrafts - these are definitely NOT green fluffy fuels, but there's no way around them for abort motors and the like.
The fuel choice for Starship is driven by what they need to colonize Mars, and there's a lot of factors involved:
The choice was between kerosene, methane and hydrogen. Kerosene is the least efficient fuel, methane is in the middle and hydrogen is the most efficient (basically, how far can one tonne of fuel propel your rocket). Kerosene is very cheap and it's also very dense, but it cokes the engines, making it harder to reuse them many times without refurbishment. Hydrogen on the other hand doesn't coke, but it is fiendishly difficult to contain, because of the very small molecule size. It's also by far the least dense fuel of the three, meaning the tanks would have to be much larger (and therefore weigh more, giving less efficiency). Methane is middle of the pack: dense, but not so dense as kerosene; cokes, but not as much as kerosene, it's cheap and it's far easier to store and handle than hydrogen.
Methane can also be produced on mars using the sabatier proces, which is a significant pro when you want to colonize mars!
All in all, Starship and the Falcons do not reduce CO2 in any way, but they are an infinitesimal part of the global CO2 emissions.