The current target is to reduce net debt to about 20 per cent of GDP by 2020; it is expected to settle at 24.3 per cent by the end of June.
On the issues of tax cuts, Joyce said the Government remained committed to reducing the tax burden "and in particular the impact of marginal tax rates on lower and middle income earners, when we have the room to do so."
After his speech, Joyce emphatically ruled out cuts to tax rates in the May 25 Budget but hinted there would be some adjustments in thresholds to address bracket creep.
Joyce also hinted at greater use of public-private partnerships (PPPs).
He said that the boost to capital expenditure would represent the biggest addition to the Government's capital spending in decades.
"To put that into context, the net new capital allocated in the last four Budgets was $4.8 billion, of which $4.1 billion was funded through the proceeds of the mixed ownership model programme."
In between Budget 2016, the Government was forecasting just $3.6b in new capital spending between Budget 2017 and Budget 2020, which would now be $11b in new capital spending.
"If you add the Government's budgeted new capital investment together with the investment made through baselines and through the National Land Transport Fund, the total is around $23 billion over the next four years, or an average of nearly $6 billion per year."
"And we want to extend that further, with greater use of public-private partnerships, and joint ventures between central and local government and private investors."
Joyce said New Zealand's stronger economic performance flowed through to the Government books.
For the first eight months of the current financial year, tax revenue was nearly 4 per cent ahead of predictions in Budget 2016.
"Our surplus in the eight months to February was $1.4 billion. That's more than $900 million more than was predicted in the [December] half-year update.
"A growing and more resilient economy allows us to meet some of the pressing needs that the Government is faced with from time to time."
But he said one of the biggest risks to the New Zealand economy were the "more insular" economic policies being pushed overseas and by the Government's political opponents domestically.
"Many politicians, even those in New Zealand, want to be more protective on trade, slash immigration, reduce foreign investment, institute radical new environmental regulation, centralise wage bargaining, blow up our R&D [research and development] incentive system or stop much-needed roading being built, and increase taxes.
"That's the opposite of a recipe for growth. That's a recipe for stalling growth."
Labour Finance spokesman Grant Robertson said Joyce had belatedly come to the realisation that everyone else had a long time ago, that the country needed an urgent and enormous catch-up from nine years of under investment by the Government.
He said Joyce was playing with numbers because National was promising to spend only $300 million a year extra from the promise in the December half yearly update.
Labour had a plan to deal with the housing crisis, get young people into work and training and grow decent jobs with higher wages.
"That is what Steven Joyce should have been setting out."
