XT network had 'immature operational systems' - report

Telecom chief executive Paul Reynolds at a February press conference explaining the XT failure. Photo / Dean Purcell

Telecom's XT network failed because the network and supporting operations were not ready to manage the levels of traffic experienced, a report commissioned by Telecom has found.

United Kingdom-based Analysys Mason was not asked to comment on responsibility or accountability of either Telecom or French company Alcatel-Lucent, which designed, built and operated the network. It found the 111 service was aligned to international standards.

The network experienced degraded performance and failures in December, January and February. The report said the severity of the failures undermined market confidence in the network but its performance was much improved after a number of issues were addressed.

Although the traffic volumes experienced following launch were within Telecom's forecast volumes, the network was not able to manage them.

The radio network controller was the weakest link. The failure of the Christchurch radio network controller led to a partial failure of the network in the southern region of New Zealand.

As network traffic increased, as yet identified software issues, particularly radio network controller software, made parts of the network unstable in certain conditions. The issues were difficult to rectify quickly.

The report also found that the initial configuration of the network and some network build issues led to coverage variability.

``In addition, in order to maximise performance and coverage for any 3G network, continuous network optimisation is required. Due to the lack of operational system and process maturity, in particular the lack of effective network performance measuring systems, it was difficult to address coverage issues quickly,' the report said.

Some aspects of the network architecture are overly complex.

Commenting on management systems and processes, the report said there were unclear areas of responsibility and the change management process was not able to manage the high level of change introduced by a new rapidly evolving network.

Telecom offered affected customers $5 million in compensation for the problems, and has since added 27 new mobile sites and 115 tower mounted amplifiers. The amplifiers were being added at the rate of 30 a week to bring the network up to capacity.

Alcatel-Lucent New Zealand and Pacific Islands chief executive Jyoti Mahurkar-Thombre said she was pleased with progress.

More equipment had increased its capacity to handle future growth and it had been modified, she said.

``The report provides an independent validation that the steps we have taken to improve the XT network are the right ones,' she said.

Telecom chief executive Paul Reynolds said the review has been both chastening and heartening at the same time.

``Clearly some serious errors were made but the report shows that XT is fundamentally sound, that Telecom, and our partner Alcatel Lucent are now on the right track. Significant progress in improving the robustness and reliability of XT has been made.'

``The fundamental problem was that the network and its operational processes were not able to adapt quickly enough during our ambitious launch programme; we acknowledge this and accept the findings of the report,' Dr Reynolds said.

Together Telecom and Alcatel-Lucent had already completed an extensive programme of improvements and the network was now performing well, he said.

Communications and Information Technology Minister Steven Joyce said the Government's primary focus had been on Telecom's ability to provide a ``robust' 111 emergency service.

New Zealanders had to have confidence in the 111 system, and all mobile network operators had to take all reasonable steps to ensure 111 calls could be made.

``I will be reviewing closely the results of the investigations into recent network outages to assess whether any further action is required to protect the integrity of the emergency calling system.

``Beyond that, it's obvious that there have been significant issues with the XT network. Telecom's customers will be pleased to see the company moving quickly to improve the network's reliability.'

NZPA