But it becomes harder when populations get older and, ultimately, start falling.
Japan's economy has been in a slump since the early 1990s, largely because of its ageing and falling population.
Its birthrate collapsed from just over the replacement rate of 2.1 per cent in the early 1970s to 1.3 per cent in 2005. It's no coincidence that its economy has contracted over the past 20 years, and endured chronic deflation it is only now addressing with massive money-printing.
Its population is forecast to fall 30 per cent by 2060. Councils in rural areas of Japan are already closing bridges and roads they can no longer afford to maintain, as their base of ratepayers either leave or can't afford the rates.
This is a spectre now facing some councils in more remote areas of New Zealand. This week's Royal Society of New Zealand report on the implications of the latest Census made that clear.
"Some territorial local authorities will have increasing difficulty in maintaining service levels for an ageing and possibly dwindling population, not to mention burgeoning numbers of visitors and tourists," the report said.
The New Zealand Institute of Economic Research's Shamubeel Eaqub has also talked about "zombie towns" in regional areas.
Councils will have to make difficult decisions to return tarseal roads to gravel, turn off town water and let parks return to bush. They will have to keep ramping up rates to a falling base of ratepayers, while also paying for facilities for more visitors.
That may force them to levy more visitor charges. Anyone buying property in places such as Wanganui, Gisborne, Whangarei and Greymouth should look at their area's population projections before putting deposits on houses or office buildings.
There are no easy solutions, but depopulation will increasingly become a topic for town planners, investors, real estate agents, economists, homebuyers and voters.
In the even longer run, it will be something to think about globally. The UN's population division said 2014 looks like the year for "peak births" in the world, with 139 million. The global population may start falling as early as 2070.
Our economies would then have to find other ways to grow.