Same goes for all freedoms. The difficulty is in figuring out (and more importantly, agreeing) where infringement begins. People burning ornamental fires in their fireplaces on the California Coast might think they have the freedom to do that, but it adversely affects asthmatics and other groups in the Central Valley. And then there are longer term and multi-layered effects like climate change, misinformation, algorithms drawing users into social media for longer and longer, etc.There's no such thing as absolute free speech. The limit is where it infringes upon the fundamental rights of others.
It's why being an "absolutist" about anything is nonsense.