A source told the Herald one of the referees was visibly upset over the relentless yelling towards them.
He said it was disappointing that match officials were subjected to abuse.
Central Football chief executive Darren Mason confirmed the regional body was in the middle of its investigation into the two players being sent off. It will be completed by the end of the week at the latest.
Mason said there were two quite different versions of events being told by the clubs and the organisation would rely heavily on the referee's version of events to establish what actually happened.
While Mason would not go into the details of the investigation until it was completed - he confirmed the spitting claim was not correct.
Mason said while men could get carried away in competitive sport, the abuse of referees and other players was extremely disappointing.
"It's a blight on our capability. I'm not going to deny it. It's a really unfortunate thing and we need to stamp down on it and we are.
"So when there are incidents of abuse fo a referee - we act as toughly as we can."
He would not confirm whether the allegations of abuse to the referee had been substantiated as he was still communicating with the clubs.
Port Hill United president Kim Matthews said she was aware of "some words and inappropriate behaviour" at the game and confirmed the club had filed a match report.
Matthews declined to go into the details of what happened while it was being investigated, saying she had only heard one version of the events.
However, she said the club did not condone inappropriate behaviour.
A Gisborne United official, who wouldn't be named but was at the game, said the club was filing a response to the Port Hill United's match report as it contained aspects which he claimed weren't correct.
He said the two players had "challenged" each other, but that the response from spectators and the player came after untrue claims from the Port Hill coach that the Gisborne player had spat.
One of the match officials believed the coach and the red card was issued despite no one else including either referee witnessing it, he said.
The official also denied claims the referees were yelled at or that the Port Hills player was blocked from entering the changing rooms.