"It was a song that played on some excruciatingly bad puns based on the names of various states," he says.
"It was a pretty awful song, typical of the popular music at the time."
Delaware got du Fresne thinking about the other great American place-name songs - from Jackson to By the Time I Get to Phoenix to 24 Hours to Tulsa - and what kind of places they actually were.
"The American song book is full of songs about places and I don't know of any other place that's quite like that," he says.
"It's been part of that canon of American culture for a very long time."
This morphed into a plan to actually see them in person and the resulting road trip with his wife is laid out his new book A Road Tour of American Song Titles, which du Fresne says is both "a travel book about music and a musical book about travel".
Each chapter is devoted to a different hit song and the town that inspired it, exploring the history of the town as well as the music. The southern states feature heavily, for good reason - they're the musical heartland of America.
"Not a lot of great songs came out of Pennsylvania and Minnesota or Indiana or places like that. But you go down to the south - Mississippi, Louisiana, Missouri, Kentucky, Tennessee, Texas - they are incredibly musical states. You can't go there and not be aware of how important music is."
Many of the towns visited are far off the tourist track and while Du Fresne didn't have major expectations, there were still surprises to be found.
"One of the songs I write about by the Everly Brothers - it wasn't a big hit - it's a song called Bowling Green and it's about a place called Bowling Green in Kentucky.
"I conjured up this really idyllic image in my mind of this place ... so I went there with high expectations and found that it really wasn't anything like what I'd imagined it to be."
However, Mendocino in northern California - made famous in a 1969 Sir Douglas Quintet song of the same name - turned out to be an absolute delight: "a romantic kind of place in a very dramatic location".
"The northern California coast is very rugged and you've got these wild seas and sheer cliffs, as well as forested mountains behind.
"Here's this tiny town called Mendocino clinging to the clifftops. It became quite a hippy retreat in the 1960s, which is why Doug Sam, who wrote the song, got to know the place."
A pair of New Zealanders showing up in obscure American towns could also be quite a surprise for the locals. This was the case when they arrived at an information office in Saginaw - a Michigan town immortalised in Lefty Frizzell's 1964 song of the same name.
"The woman behind the counter was completely taken aback. I'd say we'd probably been the first people she'd seen there for months, if not years.
"Nobody goes to Saginaw, it's just not that kind of place. She was astonished that someone from the other end of the Earth would turn up."
During the road trip, Du Fresne found many of these place names were chosen simply for the way they sounded, rather than for any deep, geographical reason.
Jackson - made famous by Nancy Sinatra and Lee Hazlewood, then Johnny Cash and June Carter - was one such song.
"I don't know if the guy who wrote the song had even been to Jackson, he just liked the sound of the name and it fitted the song," Du Fresne says.
"And 24 Hours from Tulsa could have been 24 Hours from Anywhere - it just sounded good".