Exactly. The real debate is on which parts should be off limits.

Most people can think of some speech that they consider so horrible that nobody should be allowed to say it.

People often try to hedge that position by arguing that they’re not even really infringing on anyone’s speech because their form of restriction doesn’t meet a sufficient threshold of censorship.

Does anyone?

The closest I can think of to “real free speech absolutists” is the old-school doctrinal libertarians. Even they have limits on what they believe should be allowed and specifically state that contracts should be legally enforceable.

That sounds like a much more modest proposal.