This is based on research that has shown saturated fats found in whole-fat products raises your LDL cholesterol, a marker of heart disease.
However, past evidence has suggested there are a number of nutrients found in dairy products including calcium, potassium, magnesium, vitamins K1 and K2 and probiotics (in yoghurt) that could contribute to a healthy diet.
"Diet is one of the most important modifiable risk factors of chronic disease," lead author Dr Mahshid Dehghan, a senior research associate at McMaster University, said.
"And because dairy has magnesium, potassium, all these vitamins, we wanted to see if there was a link between consumption and chronic disease."
For the study, the team looked at more than 136,000 people from 21 countries who had participated in the Prospective Urban Rural Epidemiological (PURE) study.
The PURE study looks at environmental, societal and biological effects on obesity and chronic health issues including heart disease, diabetes and cancer.
The participants, who were between the ages of 35 and 70, were divided into four categories: no dairy, less than one serving per day, one to two servings per day, and more than two servings per day.
A serving was determined as being equivalent to a glass of milk, a cup of yoghurt, one 15 gram slice of cheese, or a teaspoon of butter.
Researchers noted the dietary intakes of the participants at the study's start in 2003 and followed them for about nine years.
The high intake group, which had an average of three servings per day, was found to have lower rates of death due to cardiovascular and non-cardiovascular causes, heart disease and stroke compared to the no intake group.
Additionally, those who only consumed whole-fat at a higher intake - around three servings per day - had lower rates of death and heart disease that those who less than 0.5 servings of whole-fat dairy per day.
The team also found that the highest dairy consumption was in North America and Europe, where participants had four or more servings per day.
Consumption was lowest in southern Asia and Africa, where participants ate an average of less than one serving per day.
"There are many factors that affect the pattern of consumption, because the food culture in various countries is different," Dr Dehghan said.
"But we see the same trends. No matter, which country we looked, the trends were the same."
She says more research is needed as to why dairy is linked to low levels of heart disease as well as looking into overall diet quality in different countries.
"What I really want to emphasise is that consumption shouldn't be discouraged but encouraged especially in low-income countries and even in high-income countries where consumption is low," Dr Meghan said.
"We are not saying people eating seven servings of dairy a day should increase their intake, but that three servings - moderation - is good for you."