But today he emphatically ruled out tax cuts for the first time, speaking to reporters after a speech to the Wellington Chamber of Commerce.
He said he was close to making a final decision "but I've said all the way through it won't be something dramatic."
Some of the thresholds were a concern for him "but it's really about what we can do on Budget day, if at all."
"We are not planning to do a reduction in rates this year ... if we were to do anything it would be something around thresholds."
He lifted the capital allowance in next month's Budget from the $3 billion announced by Bill English in December to $4 billion - $812 million of the increase will be used to repair the quake-damaged State Highway 1 north and south of Kaikoura.
The $4 billion spend is a 344 per cent increase on the $900 million earmarked for 2017 in last year's Budget.
He lifted the capital allowance over the next four years from $9 billion announced in December to $11 billion, which is a 205 per cent increase on the $3.6 billion earmarked last year for the four-year plan.
Joyce said part of the current high levels of immigration are people involved in building much needed infrastructure but he thought it had peaked.
"That's one of the reasons we have migration at the moment, to make sure we fill the roles and build the infrastructure and the buildings that this country needs," he said.
He rejected a suggestion that high levels of immigration had created the need for more infrastructure.
"The spend is primarily created by the fact that we have a growing economy, which is an attractive place to work and live, much more attractive than historically when we used to have 30,000 odd net leaving each year, and that is creating the desire for growth.
"And it is a good thing. You don't get projects like the Kapiti expressway and the western ring route unless you have the capacity to grow and pay for them."
Details of the infrastructure spend would be laid out in the May 25 Budget. He said he wanted to see greater use of public-private partnerships, and joint ventures between central and local government, and private investors.
He said Amy Adams would be making housing development announcements before the Budget.