"Yoghurt, for example, has very little lactose because the good bacteria breaks it all down. Hard cheese is also very low in lactose. Parmasen or cheddar has just 0.1g of lactose per 40g serve, compared to 12g of lactose in a 250ml glass of milk."
The CSIRO study showed more women are avoiding milk and dairy foods than men, despite dairy foods being especially important for women - who are at a higher risk of osteoporosis.
"In terms of nutrients, it's not just calcium that's valuable from dairy," Dr McGrice said.
"Protein and iodine are also from dairy, as well as Vitamin A and D, Riboflavin, Vitamin B12, and Zinc.
"All of those nutriments are really important for healthy blood, immune system, muscle and nerve functionality, as well as healthy skin, growth and repair."
The trend of cutting out certain food groups, without consulting a dietitian or medical advice, has left experts involved with the study "concerned".
"The scale of people restricting their diet without a medical reason is very concerning in terms of the public health implications, especially for women," Bella Yantcheva, CSIRO's behavioural scientist, said of the findings.
"It means there is potential for nutritional deficiencies or imbalances, or the risk that an underlying health condition could be going untreated."
Dr McGrice said the diet "trend" of using plant-based milk alternatives was also having an impact, with many consumers opting for products that fail to meet acceptable nutritional values.
"If people are using dairy alternatives, they need to make sure what they are choosing to consume is calcium fortified," she said.
"The product should be at least 100mg of calcium per 100ml consumed. There's a lot of brands of oat, rice, soy and almond milk that don't have a great deal of calcium added."
Dr McGrice said people who say they were consuming other food groups to make up for their calcium requirement needed to be correctly informed.
"One cup skim milk provides approximately 375mg of calcium, while one cup of broccoli provides around 30mg calcium," she said.
"So you'd need to consume 12 and a half cups of broccoli to get the equivalent amount of calcium of one cup of milk."
While intolerances are real, Dr McGrice is concerned the dairy elimination diet has "become a trend," when in fact people can get over their problem.
"Intolerances usually come and go, and they can sometimes be developed after certain gastrointestinal problems," she said.
"Often dietitians do a wash out period if someone has an intolerance, but then bring that food back in and build it up slowly.
"Anybody who has a lactose intolerance, tend to cut the food group out entirely, when really they should be bringing it back in slowly with small doses in your diet to try and minimise the intolerance.
"It's quite simple, if people have a problem, they should see a dietitian instead of just cutting it out entirely."
According to the Australian Dietary Guideline, most people need to be consuming at least two or three serves of dairy on a daily basis, such as milk, cheese or yoghurt. One 250ml cup of milk, for example, is equivalent to one serving.
