It said expanding the primary curriculum is an unnecessary distraction at this time.
As well as this, NZ First wants to develop a similar curriculum progression document for New Zealand Sign Language, another official language.
The bill requires the Government to set 10 priority languages – likely to include Mandarin, Spanish, French, Japanese, Korean, Pacific languages and possibly Hindi as well as official languages Te Reo Māori and New Zealand Sign Language.
It also requires the Government to resource the provision of those languages in primary and intermediate schools.
Hipkins said there was real value in second-language learning.
"Kids who do a second language generally tend to do better in their first language," he said.
He welcomed the opportunity to have a discussion about what was taught in schools, including language learning, on a cross-party basis rather than being divided along party lines.
The NZ First statement said Kaye had nine years in Government to promote this bill.
"Apparently she couldn't convince her caucus or cabinet colleagues to support it, even though she herself was the Minister of Education in 2017.
"We also note recent international reports from December 2017 that show New Zealand's score in an international English medium reading test for 10-year-olds has dropped for the first time in 15 years."
These results were from 319,000 children in 2015 and 2016 and are part of the legacy of the previous National government, it said.
"These results show that a focus on our student's English abilities should remain a priority."