"I wasn't working, I was a solo dad on a benefit. I wanted to stay with my boy until he was 6 years old. I had no ambition to work."
His life changed when a resident of Randwick Park, Dave Tims, asked him to help develop a new skateboard park for the 1970s housing area through a scheme called "Urban Neighbours of Hope".
Mr Brown knew what skateboarders wanted and had often designed his "ideal" park on PlayStation's Tony Hawk's series.
"I said I would help him and Dave said, 'well, you will have to stop drinking at the skatepark'. Me and three other skaters worked with the designer. It was a dream."
The skatepark opened 18 months ago, twice the size of the original, and is a centrepiece in a $5.3 million staged improvement in facilities.
"Back in the day, you would not have gone to the Randwick because it was a place of rebels, where everybody was wasted and you didn't fit in unless you were drinking.
"Now, we don't allow smoking, drinking or using rough language. You can't wear bandanas or patches.
"The culture has changed to be more polite though we are still expressing ourselves."
This is Mr Brown's third year as an inclusive youth worker at Alfriston College, another role that he never thought was possible.
"The key was that I had relationships with the youth in the community.
"They are comfortable talking to me about anything and it makes my job easier."
The job got harder when he was asked to take a class for science.
As a student he was bullied and hated school and did not pay attention.
"That term I found it hard trying to keep the class excited and turning up.
"So I used the science of skateboarding and every second week we would go to skate together and try to put what we had learned on the pavement."
His work extends beyond the school gates to after-hours calls from distressed youth and sometimes parents.
"I have young people come over home and I talk to anyone at the skatepark who wants to.
"Everybody needs a helping hand.
"The guy who approached me at the skate park was my hand and that's all I needed to get on my feet and find out what I was good at.
"And I found I am good at this job."
TEDx
• A world-wide movement, encouraging local events to present residents' ideas about helping others to fulfil their potential.
• Some of the 10 speakers at the event are called "bright spots" by sponsor Ko Awatea - Counties Manukau Health's innovation arm - because they are supporting South Auckland residents to live well.