"Because of these costs, the Government has decided to remove the $1000 kick-start payment from today," Mr English said.
He said it was a permanent change and the perk would not be reinstated at a later date.
Read more:
• Budget 2015: 10 things you need to know
• Budget 2015: How it affects you
Because the Kiwisaver scheme was proving very popular, a payment was no longer needed to attract people to enrol.
The scrapping of the subsidy would save Government $125 million a year over the next four years. The minister said this money would be reinvested into "priority public services".
The annual tax credit of $521 would not be changed.
Employers would still be required to contribute at least 3 per cent of a worker's pay packet and employees would still make their own contributions.
The National-led Government has gradually pared back the Kiwisaver scheme in previous budgets, reducing the government contribution.
Video: Economics editor Brian Fallow - This is a tight-fisted budget:
Mr English defended the latest change, saying that the employer contribution and the annual tax credit meant people would still have an incentive to sign up to Kiwisaver.
In total, the Government would spend $850 million on the scheme this year, through the ongoing cost of the government contribution to Kiwisaver, worth $521 per member, and the kickstart payments claimed so far.
In the next financial year, the Government was forecast to spend $705 million on the tax credit and $12.3 billion on New Zealand Superannuation.
Video: Finance Minister Bill English - Helping parents into work:
