He argued that problems in the crown case couldn't be brushed aside or "sneered at" and they created "unavoidable doubt".
Mr Ray said Ms Wass had told Harris what he'd allegedly done but had failed to prove it so that the jury could be sure.
He asked whether Harris could really be described as a "sinister pervert" and someone who mesmerised women.
The delay of up to 45 years between the alleged assaults and when they were reported to police caused serious problems for Harris, the court was told on Monday.
"It's much easier to make allegations like this than it is to rebut them," Mr Ray said, adding Harris couldn't win because if he failed to remember something he was accused of deliberate lies and if he did recall details they were dismissed.
The lawyer noted Harris was of good character with no criminal convictions but the prosecution had set about destroying his reputation "with vigour and enthusiasm".
Mr Ray acknowledged the entertainer was "clearly far from perfect" given he'd admitted having two extramarital affairs but insisted that didn't make him guilty of the indecent assault charges.
Indeed, the barrister said given the trial had occurred with the whole world watching, Harris had already been punished for his infidelity whether he was found guilty or innocent.
"He has been punished for his infidelity by, effectively, public humiliation."
During the trial Harris said he'd never been to Cambridge - where one of the assaults is said to have occurred in the 1970s - until three of four years ago.
Video footage subsequently emerged of him participating in Star Games which was filmed in the city in 1978.
Ms Wass accused Harris of deliberately misleading the jury but Mr Ray said if the Australian had any recollection of the event he would not have issued a blanket denial.
"No defendant would be that stupid," Mr Ray told the jury.
"Mr Harris didn't lie, he forgot."
The defence barrister said another of the four complainants may have seen someone else singing a Rolf Harris song at a community centre near Portsmouth in the late 1960s and mistakenly believed it was the Australian who then groped her when she asked for an autograph.
"She could honestly believe what she's saying but still be wrong," Mr Ray said, adding the danger with honest but mistaken victims was that they could be "very convincing".
Mr Ray also savaged the prosecution's argument that there were "striking similarities" between all the alleged assaults.
"Beware of barristers and their theories," he said.
"The prosecution has exaggerated just how compelling any of this is."
Mr Ray criticised many of the six women who have given supporting evidence that Harris harassed them in Australia, New Zealand and Malta.
He pointed out that some had kept photographs of themselves with the star.
In one case a mother who claimed Harris assaulted her daughter and then herself subsequently put a cartoon the artist had drawn on her daughter's bedroom door.
What mother would do that if she'd just been sexually abused, Mr Ray asked the jury.
The 84-year-old is charged with assaulting four girls in the UK between 1968 and 1986.
Harris denies inappropriately touching any of the alleged victims.
