The only exception he says, is when they are given out of natural love and affection such as from a mother to her son.
A sugar relationship is where an older man or woman spends large amounts of money on a younger girlfriend or boyfriend in exchange for a relationship.
SeekingArrangement, a US-based website, claims to have more than 50,000 members signed up in New Zealand.
Gifts received by sugar babies interviewed include boob jobs, a $4000 Hermes watch, $2800 Louis Vuitton bag, business class travel and regular cash transfers.
Arrangements vary, with some sugar babies saying they have sex with the men, and others keeping their contact to dates and no physical contact.
"It is quite safe to assume that in these sugar relationships, the love and affection is not unconditional," Han says.
"When the gifts and money stops coming, the love and sex will also end."
Inland Revenue last year launched an anti-cash-job campaign targeting tradespeople, and Han says the department is cracking down on the hidden economy.
"Basically, the sugar babies are committing a tax crime, and they should do the right thing and speak to their tax advisers or chartered accountants," Han added.
High-profile escort Lisa Lewis said she had to pay tax on gifts after a tax department audit following a media report in 2011.
"I had to pay tax [after] the media printed I had been gifted a boob job ... it was a gift from a truck driver," Lewis said.
Lewis said she had paid taxes on gifts that included a Tag Heuer watch and an $18,000 treadmill, and has kept up with her tax obligations.
"I only learned that I had to pay taxes on gifts after being audited ... working in a strip club then, I was clueless about having to pay taxes on these things," Lewis said.
It is her view that there is no difference in the services provided by sugar babies and escorts.
An IRD spokeswoman said it is not the department's role to "categorise anyone as a sex worker or not".
She said whether or not a "transfer of value", for example smart phone, overseas holiday or cash, is a non-taxable gift or income depends on the type of relationship the two people have.
"If the transfer of value is because of natural love and affection, it's a gift," the spokeswoman said.
"If it's in return for services or something received back, it's not a gift."
Regular cash transfers which are relied on for day-to-day living or support is also deemed as income.
"Each case will need to be looked at individually to decide if it is a gift or income," the spokeswoman added.
A tax guide for self-employed sex workers, or IR 296, based on current tax laws is available online.
