Shrem is no stranger to run-ins with the law. In 2014, he was sentenced to two years in prison for using Bit-instant, a former cryptocurrency exchange which he co-founded, to funnel bitcoin to shadowy actors seeking to buy drugs from internet black market Silk Road.
He assisted Robert Faiella, a plumber from Florida who operated as BTCKing online, with the transfer of more than US$1m worth of bitcoin to the Silk Road in 2012. Though the early bitcoin advocate came clean in the Silk Road affair - pleading guilty to aiding and abetting an unlicensed money transmitting business - he plans to stand his ground in this case.
But "bitcoin's first felon" may be running out of luck.
The Winklevoss brothers came to be worth more than US$1 billion last year after capitalising on a US$65m payout from Facebook - US$11m of which were invested in bitcoin at a 2013 price of US$120.
Shrem, who became an adviser to the twins in 2012, is alleged to have stolen 5000 bitcoin.
Brian Klein, Shrem's lawyer, says the accusations made by the Winklevoss brothers are unsubstantiated.
"The lawsuit erroneously alleges that about six years ago Charlie essentially misappropriated thousands of bitcoins," he said. "Nothing could be further from the truth."
The Winklevoss brothers claim that after just a few months of working with Shrem, they were short-changed.
In September 2012, the twins gave Shrem US$250,000 to invest in bitcoin at a price of US$12.50 at the time. A month later, the investor returned to the pair with US$189,000 worth of bitcoin.
The cryptocurrency was again trading at US$12.50, which meant approximately 5,000 bitcoin had gone missing.
Shrem failed to account for the bitcoin when confronted by the brothers, who were forced to launch an investigation into their missing assets.
Through an external investigator tracking the blockchain, a ledger over which cryptocurrency payments are made, 5000 bitcoin were found to have been moved in 2013 through Shrem-associated accounts on digital wallet platforms Xapo and Coinbase.
"When he purchased US$4m in real estate, two Maseratis, and two power boats, we decided it was time to get to the bottom of it," Cameron Winklevoss told The New York Times.
A judge in a federal-district court in New York has since approved a request by creditors for Shrem's funds held across Xapo and Coinbase to be frozen.