Budd said the chalk message was up for a few days before the able-bodied customer complained it was offensive.
A tweet showing the blackboard message attracted a backlash against the popular coffee shop, which Budd opened 13 years ago.
Commenters called it "revolting", "sick" and "crappy behaviour".
Cath Sheard wrote: "How disgusting. As a woman with a disability, I'd never go there or recommend to anyone."
Budd said it was never his intention to "offend anybody in a wheelchair".
"The thing with humour is, you try and take it as far as you can without stepping over the edge and now and again you do. It's just the way humour works.
"I personally think it's really funny but the last thing I want to do is offend anybody."
He updates the sign weekly with jokes and issues for debate and said it's a drawcard to the business.
"I do like to stir things up a little bit but I don't like to stir things up a lot. I like to get people thinking. You have to be edgy to be funny.
"We put things up there that are not necessarily our opinions but it's to get discussion happening and people talking about stuff and I always find that's valuable."
But disability advocate Kylee Black was appalled and disappointed at the sign.
Black, who suffers from Ehlers-Danlos Syndrome which confines her to a wheelchair, was appalled a cafe owner thought the message was okay.
"Because you're actually making fun of an entire population of people at their own expense where actually my wheelchair doesn't limit me, my wheelchair gives me freedom."
She dismissed Budd's explanation of pushing boundaries as not relevant or appropriate.
"It's not doing anything to create positive affirmation of awareness around disability. The only thing it's doing is actually having a laugh at the disabled person's expense."
Black said the fact the joke was also aimed at women made it worse because it exploited two minorities.
"It's misogynistic. If we're wanting to create healthy debate, and if we're wanting to create open conversations we actually need to think about the light in which we're doing that."