They also asked detailed questions about exercise habits like how often and how intensely the men worked out.
Among those who took part were running, cycling and ameteur endurance athletes who were contacted through trade magazines.
Each of the men was then categorised according to whether they had short or long workouts and if they were light, moderate, or intense. Their sexual appetite was also categorised into high, moderate or low libido.
According to the researchers at the University of North Carolina in Chapel Hill in the US, there was a "significant" relationship that emerged.
Lead researcher Anthony Hackney, a professor of exercise physiology and nutrition, said that those who had a less intense workout had a higher libido.
People who exercised for a shorter amount of time wanted more sex as well.
Professor Hackney said that there was a clear 'tipping point' at which exercise started to ruin your sex life, a point he hopes to narrow down with further research.
The study was published in the journal Medicine & Science in Sports & Exercise.
It said: "Exposure to higher levels of chronic intense and greater durations of endurance training on a regular basis are significantly associated with a decreased libido scores in men."
Normally women trying to get pregnant are the ones who face many questions about their love life and the amount of exercise they do.
But Professor Hackney said: "Fertility specialists will often ask a woman about whether and how much she exercises.
"Based on our data, we think they should also be asking the man."
This article was originally published by The Telegraph.