The court has been told that O'Brien told Elliot: "Look Mate, you're full of s...".
O'Brien has given evidence that Elliot then pushed him back about 4 metres against a display cabinet, tried to headbutt him and threatened his life.
O'Brien said Elliot had also grabbed his wife by the neck when she tried to intervene, and that he then punched Elliot.
"I hit him in the head to get him off my wife," O'Brien told the court on Monday.
Elliot then knocked O'Brien unconscious.
But Elliot said he did not intend to injure O'Brien.
"Tony O'Brien attacked me first. I'm telling you straight," Elliot said during a video interview recorded at the Hastings Police Station six weeks after the incident on 26 January 2021, which was shown to the jury on Tuesday.
"There was no malice intent. I just went into self-defence mode," Elliot told the interviewer, Constable Adam Laing.
Elliot said in the video interview that the couple's son had also raised a fist at him and his more than 30 years of martial arts training had kicked in.
"There were three of them and I was basically by myself," Elliot said.
"My only intent was to protect myself. It all happened so quick," he said.
Crown prosecutor James Bridgman said the Crown did not shy away from the fact that O'Brien threw the first punch.
But he asked the jury if the force used by Elliot in response was reasonable.
"That force must be only as much as required to respond to the threat. Self-defence does not give you free rein," Bridgman said.
"Mr Elliot's use of force was wholly disproportionate."
Defence counsel Leo Lafferty said Elliot was at the shop peacefully, lawfully and acting in a well-mannered way in a private, cordial conversation.
"We have the right … to have freedom of expression, freedom of thought. That's what Stephen Elliot was doing in a peaceful, lawful manner."
The jury has been shown photographs of injuries sustained by the couple: O'Brien had a black eye and scratches to his face and chest, and O'Toole had a fingernail mark on her neck and bruising to her arm.
The first two police officers called to the shop both said they saw swelling on the left side of Elliot's face.
The jury of six women and five men is expected to retire after Judge Russell Collins sums up on Wednesday morning.
One juror has been discharged.