According to the Swindles, the money loaned to Vintage was advanced on the basis that Withers would manage the funds and ensure they were used correctly, and a net loss of $2.62m was caused by Withers negligence.
A number of conditions associated with the loan were outlined to the court.
This included Withers saying he would be a signatory for Vintage's cost account and that he would ensure the funds in the account were used correctly.
Justice Mary Peters, who heard the case, said Withers' statements "were false".
Justice Peters said instead of the Swindles' funds being used for costs as they arose they were immediately transferred to related companies.
Withers was found liable under the Fair Trading Act for misleading and deceptive conduct.
The judge reduced liability to 50 per cent of the Swindles losses saying they were equally to blame.
The court heard funds for the wine production were raised through the brokerage of Kingsley Turner, who acted as the Swindles' agent, and managed the loans to Vintage and acted on their behalf.
Justice Peters said Turner, who received financial statements for the companies annually, would have seen that the firm was making intercompany loans to its other businesses and was therefore in breach of its loan agreement.
The court also heard in 2008 Turner was advised that a Vintage company would default on its repayment later that month.
Turner arranged bridging finance, which was repaid, but did not inform the Swindles of the default or the additional funding.
Finally, the Swindles did not appoint receivers until December 2010, around 14 months after Vintage's repayment default, a time frame the court said was excessive.
Based on this, Justice Peters said the Swindles should have been more vigilant and, as Turner was acting for them, they were equally to blame.
The Swindles appealed the decision to the Court of Appeal, making the case in July last year that they would not have made the loan advances without Withers' undertakings and advice, therefore he should be liable for the full amount.
This appeal was dismissed by Justice Rhys Harrison in a recent judgement, based on several factors which Justice Peters highlighted in the earlier High Court decision and ruled Withers would have to repay half of the money lost or $1.16m.