I would argue we have gone the other way: we are so busy stomping down Nazi like ideologies that the stomping it down has become as bad of hate speech as the Nazis were.
Until the government begins a mass extermination program for haters, I can't help but feel this is a hyperbolic argument. Not granting permits to march is totally incomparable with genocide.
If we're going to tolerate Nazi and Nazi-like speech, don't we also have to tolerate those who would should them down?
Until we start rounding up Nazis, and Nazi sympathisers, and politely asking them to reside in this here concentration camp and eat this rotten-fish-head-with-cabbage soup, shouting them down seems quite reasonable.