"Your lowest risk of death is if you are physically active and don't sit," Professor Bauman said.
"Your highest risk is if you don't do any physical activity and you sit a lot of the day.
"If you think about a few hours of TV every day and driving to and from work and at work mostly sitting, it's not hard to get into that 10-11 hour-a-day range.
"What we have to do is work quite hard to undo that."
Prof Bauman said it was vital that people who sat for most of the day incorporated extra physical activity into their routines.
"People doing high amounts of physical activity, and that's an hour a day, are mostly offsetting the effects of sitting but few of us can get to that and struggle to reach the Australian recommended levels of half an hour (of exercise) a day," he said.
"The more you sit in the day the more you should try and build in a few extra minutes of physical activity by walking to the bus stop or to the sandwich shop at lunchtime and using the stairs instead of the lift."
Prof Bauman said scientists had not pinned down all the reasons why sitting for prolonged periods of time was so bad for people's health.
One theory, though, is linked to the effect sitting has on blood-sugar levels.
"What's happening is when you sit, the meal you have just eaten is broken down into sugar and your blood sugar stays high," Prof Bauman said.
"Sugar wants to be taken into muscles and the liver to be used but if you're sitting it's just circulating so your blood sugar stays high."
The findings will be presented on Thursday at the annual meeting of the 45 and Up Study, the largest ongoing health research project in the southern hemisphere.
The study began in 2006 and tracks the health of 260,000 men and women in NSW.
-AAP