Budgen, 47, a lance corporal in the Royal Welsh Regiment, is accused of attacking the woman when he was 18 on the day he played a rugby match in Rotorua.
Police took DNA samples at the time, but were unable to make a match and the Mirror reported that police also travelled to the UK to take a further sample in 2015 after advanced testing flagged up a potential link.
Budgen, who was born in New Zealand, had served in the NZ Defence Force before moving to the UK in the 1990s.
There, he served with the Royal Welsh regiment in Northern Ireland, Iraq and Afghanistan.
Budgen also played professional rugby with the Northampton Saints and was a key member of the Exeter Chiefs' successful 2010 RFU Championship side.
He became the oldest try scorer in Premiership rugby history in 2012 when he scored for Exeter Chiefs 78 days short of his 40th birthday.
Budgen retired from professional rugby in 2014.
The Mirror reported that he was held by Scotland Yard's Extradition Unit at his Hampshire barracks in December.
As part of his bail conditions, he agreed not to go to any international travel hub and must report to a police station every weekend.