And this is where it gets tricky.
If we are happy to punch people espousing Nazi ideology, who else are we allowed to punch? And perhaps more importantly, when can people punch us?
My grandmother was no fan of Nazis but she was pretty damned racist. Her views were not as extreme as Spencer's, though, so perhaps we punch her but not so hard? In fairness, this is moot. My grandmother is long dead.
How about abortion? Many people see abortion as the murder of a child and an affront to God, are pro-lifers cool to start cracking those who argue in favour of a woman's right to choose?
When we can punch, how hard we can punch? If only we had a set of rules that could govern these things. And of course we do, it's called the law.
Western democracies allow violence in very prescribed ways. You can use force to defend yourself and your property in New Zealand, for instance. You cannot use force because you don't like the views people espouse, regardless of how odious they are.
In fact, we enshrine the right to free speech and celebrate it. Voltaire's principle states: "I disapprove of what you say, but I will defend to the death your right to say it".
Apparently, that doesn't make memes as good as those using action hero Indiana Jones and the cartoon character Captain America punching Nazis. These have been widely used to support the deeds of Spencer's assailant. A writer of Captain America comics was unimpressed. He tweeted "Cheering violence against speech, even of the most detestable, disgusting variety, is not a look that will age well."
It's almost like we shouldn't calibrate our moral compasses from cartoons. That's a setback.
Spencer says he fears the video will haunt him forever and that he will hate it. Given the amount of hating he already does, I'm tempted to think he's reaching capacity. And that brings me to a rather practical point.
I don't think punching him will make him change his views. It may actually help entrench them if the gang theory of cohesion through conflict applies, which I suspect it does.
But even if it did positively impact on his thinking, a society that solves its problems with arbitrary violence is not one I would fight to save. Our society should be shaped by ideas and not violence. Our commitment to the principles that underpin free societies are not challenged by easy issues, they are challenged by hard ones. When we feel at our most pained is exactly when we must strive to hold the line.
The idea of losing civilised values to counter uncivilised ones has been aimed at those conducting the war on terror with regard to torture and drone strikes and the like. It's equally applicable here.
But having made no claims to being perfect, I'm not immune to indulging in instances of violence-inspired schadenfreude. That kid who watched Buck Shelford whack a Welshman still exists.
I admit to appreciating the aesthetic of the punch that collected Spencer, but I recognise that as a guilty pleasure that mustn't be nurtured.
Such violence ought to be condemned and condemned strongly, even if it runs against our instincts. Unconvinced? How about I add: I feel certain that's what Indiana Jones would do.
