Andrew visited Nygard at his lavish estate in 2000, after the Canadian fashion designer had agreed out-of-court settlements with three employees who accused him of sexual harassment.
The prince and his ex-wife Sarah Ferguson stayed at Nygard Cay, near Nassau, with daughters Beatrice and Eugenie.
The new case threatens further embarrassment for Andrew, who stepped back from public life last year following a backlash over his relationship with paedophile financier Jeffrey Epstein.
According to a 99-page lawsuit filed against him in New York, Nygard used his ten-bedroom mansion to host "pamper parties" for teenage girls recruited at shopping malls and fashion events.
He and his employees told the girls they could be fashion models, but then plied them with wine and drugs before Nygard allegedly raped them.
Prince Andrew's been connected to yet another sex scandal involving Peter Nygard. Photo / AP
Three alleged victims were 14-year-old virgins and said they felt scared and humiliated by the attacks, which left one needing medical treatment.
The case has been brought on behalf of ten women who allege they were raped by Nygard.
Eight were teenage girls and two were adult employees on the Nassau estate.
Lawyers said Nygard, who is worth an estimated £690million, ($1.4billion NZD) ran a "decades-long sex trafficking scheme" that destroyed innocent lives.
They said he used his company's private jet, dubbed "N-Force", to take his victims with him on visits to London, Germany, Italy and China, and treated them as his "personal servants".
Two yachts were allegedy used to transport drugs and liquor for the parties and Nygard was said to keep a database of 7,500 potential victims.
The case has parallels with some of the accusations against Epstein, who used his private jet, known as the Lolita Express, to traffic young women and girls.
One Epstein victim, Virginia Roberts, claims she was trafficked to London and forced to have sex with Prince Andrew when she when she was 17.
Andrew has repeatedly denied her account and he is not involved in the Nygard lawsuit.
A spokesman for Nygard said the allegations were "completely false, without foundation and are vigorously denied".
