The building is the old palace for the Avignon pope. In 1309, the new Pope Clement V fled there from insurgency in Rome. It marked the creation of a new era of dual Popes and hosted six papal conclaves in its time. It slipped into obscurity when the church reunited in 1370 and popes returned to Rome.
And in the Grand Chapel, where popes communed with God, I'm looking at an orange vagina.
"Only the French," I smiled, "would have so much cheek."
Of course the guestbook was full of comments about how disgusting, sacrilegious and disrespectful the art was. It seemed that I, and one wise-cracking Australian, were among the only people who liked it.
I could just imagine the palace's curator being approached by some unshaven, chain-smoking French art dealer in linen trousers, flogging paintings of naked women. When they said yes, what was going through their mind?
I reckon it was a cheeky, rebellious jab at protocol; a reminder that the French didn't play by anyone's rules.
I know another country that's famous for its supposedly rebellious attitude. But can you imagine Auckland cathedral's reaction if we tried to put up paintings containing pubic hair?
The howls of protest would be heard in Darwin.
It's a shame. One of my favourite parts of the New Zealand psyche is the playful disrespect for authority. We're supposed to be the country which casts a critical eye over "the way we do things" and say "nahhh, I don't reckon we'll do that, eh?"
It's the attitude that brings out the best in New Zealand; it saved us from the horrors of nuclear power and the English class system.
It's also the reason why, while queuing for two hours for Versailles station's one broken ticket machine, I could pass the time bitching about French customer service with the guy from Mt Albert behind me.
But seriously, having just come from England, a country where pointless routines are a national pastime, New Zealand's fun rebelliousness is beautiful. It evaluates and preserves the good in a situation, whilst stripping away the rot protected by hideous "tradition".
But if Auckland tried for a cathedral's coloured clitoral collection, there would be cries of havoc.
Primarily because our rebelliousness is only really deployed reactively, not proactively. It'll appear when a specific moral issue is thrust on to us. But this art is an ode to the general principle of plucky rebellion.
And we're much shakier when it comes to rebelling without a specific cause. The idea of rebelling to celebrate the spirit of non-conformism makes us feel uncomfortable. It rubs against our other deep vein of conservatism. We don't want to rock the boat - unless we really have to.
But our size, history and geography put us in a unique position to be the charming rebel, and we should take every opportunity to reinforce our strength in this role. It's what gives a voice to causes that actually matter.
Other, bigger countries can't be the daredevil outsider. They've got commitments to UN rules, historical feuds or leaders in court for underage sex.
They're more worried about keeping everything running smoothly than making a principled stand.
So we shouldn't react with disgust like the tourists in the guestbook. We should smile rakishly and admire France's respect for humour, sass and sticking it to the man, man. Because New Zealand, Australia and France are the only countries who could do such a thing. And if being rebellious is a competition between us three, then we know who needs to win.