"Essentially [the loss] forced us to change or review a lot of decisions we've made in relation to quality of life for our family."
Mr Fuller, who had no inkling the investment statements he received were false, had not heard from Chalmers since his offending was discovered and had not received an apology.
Chalmers' convictions stem from trading foreign currency and misleading banks when buying three Bay of Plenty properties.
When obtaining loans for these, the Pukekohe teacher, who used to live in Tauranga, showed banks accounts from his company, Chalmers Cameron Investments, passing off the balance as his own money, not that of investors.
He also made 519 false statements to 64 investors about his company in which its parlous financial position was not disclosed.
"By August 2011, your false reports claimed investments held of $3.64 million when in fact you held $140,000," Judge Ron Ronayne told Chalmers in court today.
"By March 2012 you'd inflated the claimed investments to $6.2 million, when in fact it held $290,000."
Chalmers initially received money from his family members and friends and then from colleagues and acquaintances, promising a return on the investment. Despite not delivering, he took money for fees to which he was not entitled and even took money his sister deposited in his bank account for her portfolio.
"It's clear to me that you lied to investors when that money was largely gone, and in the last few months you conned for investors to put their savings into your business," the judge said.
The total loss Chalmers caused investors is disputed, but is about $1.5m. Banks were conned out of $1.4m, but as they held securities that was repaid.
Defence lawyer Paul Mabey, QC, said Chalmers was out of his depth trading foreign currency at night while working as a teaching adviser in the UAE.
By his own admission, Chalmers was incompetent at forex trading but when the operation was set up there was no intention to deceive investors, Mr Mabey said.
Mr Mabey said Chalmers rejected assertions made by some who lost money through him that he lived a lavish lifestyle in the UAE.
SFO director Julie Read said the prosecution of serious financial crime is important to maintaining a confident economy.
"People must be confident about investing to create economic growth which is why the SFO places such importance on the success of prosecutions such as that of Mr Chalmers.