According to defence lawyers, the man and the girl met in a park and the girl voluntarily followed him to an apartment and consented to have sex. They also claim their client, who was 28-years-old at the time, thought the girl was over 15.
The girl's family filed a complaint for rape in the town of Montmagny but prosecutors reportedly felt the suspect did not use violence or coercion.
According to French law, rape is defined as sexual penetration committed "by violence, coercion, threat or surprise".
"She was 11 years and 10 months old, so nearly 12 years old. It changes the story,'' defence lawyer Marc Goudarzian said on Tuesday.
"So she is not a child.''
His colleague Sandrine Parise-Heideiger went further, claiming "we are not dealing with a sexual predator on a poor little faultless goose".
She said as soon as children have "sexual expressiveness and you have an attitude of putting yourself in danger'' then "it doesn't necessarily mean the person on the other side is a sexual predator".
Children's rights groups and a psychiatrist testifying in the case disagree.
Carine Diebolt, the lawyer for the family, asked the court on Tuesday to re-characterise the charge from sexual abuse to rape.
The suspect "knew very well she was a young child", said Armelle Le Bigot Macaux, president of the Cofrade, an umbrella group for children's rights.
"This young child isn't protected today by our French society."
If convicted of sexual abuse, the suspect, a father of two, faces up to five years in prison. The rape of a minor under 15, however, is punishable with up to 20 years in prison.
The presiding judge said the prosecutor chose a wrong charge and ordered the case to be sent back to investigators for a thorough investigation. As a result, the trial was postponed.
"It's a victory,'' Diebolt told reporters after the trial.
"The main thing is that [the girl] can at last be heard as a victim of rape ... we can say it's a victory for the victims.''
Goudarzian, the defence lawyer, claimed the court decision was a result of the wide publicity given to the case.
The Montmagny case is one of several that have prompted an uproar over France's rules on child sex abusers, which are considered too lax by child rights groups and feminists.
French President Emmanuel Macron's government has proposed a bill to introduce a minimum legal age for sexual consent for the first time - and includes a provision saying that sex with children under a certain age is by definition coercive.
The proposed minimum age hasn't yet been decided on, but the cutoff could be between 13 and 15.
The bill, a broad-based measure aimed at fighting "sexual and sexist violence", is expected to be presented to the French Cabinet next month.
A similar recent case caused disbelief and outrage.
A French criminal court in November acquitted a 30-year-old man accused of raping an 11-year-old girl in 2009.
The jury in the Paris suburban region of Seine-et-Marne found that he didn't use violence or coercion.
The subject of sexual misconduct has drawn fresh attention in France and worldwide since rape and sexual assault allegations were made against Hollywood film producer Harvey Weinstein.
- With AP
