The infamous Lambing Flat Riots in July 1861 - during the gold rush - are credited as one of the key reasons the White Australia Policy was introduced, says Brian James, a committee member, from the Lambing Flat Folk Museum.
Thousands of Australian miners drove the Chinese out of the area during the riots and the museum still has The Roll Up Banner, which was hand painted with the Southern Cross emblem and the words "roll up, roll up, no Chinese" more than 130 years ago.
The gold rush also brought booty for bushrangers.
Ben Hall was the area's best-known bushranger, carrying out around 600 different robberies over a period of five or six years. But Frank Gardiner was responsible for Australia's biggest gold robbery, which took place about two hours from Young in Eugowra.
"Frank Gardiner was the brains behind the escort robbery near Cowra and the amount of gold [he stole] on gold prices today it would be worth something like A$30 million [NZ$39.4m]," James says.
"It was one of the biggest robberies of the time but quite a few of the bushrangers involved lost their gold and made very little out of it."
Gardiner himself was one of the few bushrangers that didn't end up dying in prison or being shot or hanged.
"There was a lot of bushranger activity here but it wasn't as dramatic as Ned Kelly," James says.
Visitors to the area can try their luck on the old goldfields but James doesn't fancy their chances of getting rich.
"The museum lends out gold pans if you want to try your luck ... I am not going to say you will find any gold but you can try."
Rather than gold, today visitors flock to Young for fruit-picking and eating.
The district supplies 60 per cent of the country's cherries, amounting to over half a million cases a year.
A cherry festival is held in the town every year and many of the orchards around Young offer a "pick your own" experience.
Once you have worn yourself out in the goldfields or orchards there are plenty of places to re-fuel ranging from local pubs with hearty country meals to a fine dining experience at Mullino's bar and grill.
Young's Commercial Hotel is a popular lunch choice for visitors, but be warned, while the food is good the servings are large enough for a hungry miner!
Accommodation in Young ranges from farmstays to four star accommodation at the town's Federation Motor Inn.
The town itself, which is home to about 10,000 and growing by about one per cent a year, offers a range of shops selling everything from jams and other cherry-related merchandise to fudge, fabric and wool.
So let your sweet tooth, gold fever or bushranger spirit guide you there.
- AAP
Getting There
Fly there with Air New Zealand Book now
Find out more at Australia.com