"The review has uncovered some deeply concerning shortcomings, including a failure to carry out the survey as it was designed, no training manual for survey interviewers, insufficient supervision and monitoring, and a lack of interaction between the survey company, MBIE, and Stats NZ, which has allowed sampling and data processing problems to build up,'' said Roberts.
Understanding markets was essential for tourism development. Issues with the IVS flowed through all industry expenditure data and into the wider government accounts.
There was an alarming lack of oversight from government officials.
"The problems with the IVS have undoubtedly shaken the tourism industry's confidence in MBIE data, and it's a concern we have heard time and time again from industry members. This needs to be fixed and TIA will continue to engage closely with MBIE and Stats NZ," said Roberts.
In a summary of its review, Stats NZ said the survey was generally fit for purpose, but some key areas need attention.
Some technical features of the survey need to be implemented more effectively so that the survey can deliver with confidence the key statistics that it was designed for.
These technical shortcomings have emerged, largely because the survey has not been kept up to date with ''real-world changes'' and the main parties to the survey (Ministry of Business, Innovation and Employment, Kantar TNS and Stats NZ) have worked too much within their respective silos.
''The loss of trust and confidence of some stakeholders in the survey stems also from shortcomings in the way the findings of the survey have been communicated to them and the manner in which they have been engaged.''
Stats NZ made 10 recommendations:
1. Improve the survey design documentation and metadata
2. Adjust the survey design to real-world changes
3. Improve the sample allocation and respondent recruitment processes
4. Improve the online questionnaire
5. Improve editing and imputation systems and monitoring
6. Work together better
7. Improve governance of the survey
8. Improve engagement with stakeholders and customers
9. Improve explanation of the survey findings
10. Assess the combined effect of the recommended improvements to the survey processing system before deciding to revise historical series.
MBIE said that although there are areas that need attention, the review did not ''currently'' recommend any revisions be made to the historical series.
The main concern of key stakeholders relates to the credibility of the reported spending pattern of international visitors in 2015-2017 but the ministry said the review had found no evidence to discount the reported pattern over that period.
''MBIE is taking this opportunity to refine processes to ensure the information delivered on international visitors to New Zealand is accurate and valuable.''
