New Zealand bandwidth pitiful says grid guru

By ADAM GIFFORD

From his desk at the University of Chicago's computer science department, Professor Ian Foster can move a terabyte of data to Switzerland in 20 minutes.

In New Zealand last week to speak at development conferences, it took the ex-pat Kiwi four hours to send a 1 megabyte file out of the country, highlighting the challenges of New Zealand taking a prominent role in the world's knowledge economy.

"Even if people want to collaborate on a global space, they can't," said Foster, who left New Zealand 25 years ago with a degree in computer science and is now a world authority on grid computing.

His current expertise came out of his interest in working out how advanced technologies can be used for problem-solving in the sciences.

Take the amount of data produced in heavy-ion collision experiments at Cern, the European Laboratory for Particle Physics in Switzerland.

By 2005 detectors at the lab will be producing several petabytes of data a year, requiring sustained applications of some 20 teraflops or floating point operations a second to do even rudimentary analysis.

The largest contemporary supercomputer is capable of 3 teraflops a second. Rather than build a bigger supercomputer, it is likely the data will be analysed by drawing on the processing power of thousands of computers scattered across the internet.

There are already several such projects. Since 1997 the Entropia network, with more than 30,000 computers, has been tackling major scientific problems at 1 teraflop a second, including the search for the largest prime number.

The SETI@home project links 500,000 PCs in a search for evidence of extraterrestrial intelligence in data from the Arecibo radio telescope.

"What we are about in grid computing is to enable collaborative teams to pool resources and apply those resources to challenging problems," Foster said.

Just having access to a computer and a network isn't enough. Global collaboration requires relationship building. "The internet allows us to sustain and develop collaborative relationships, but we need the initial human contact," he said.

Foster hasn't written New Zealand off as a contributor to such collaborations, despite the distance. The bandwidth is there on the Southern Cross cable, even if current pricing and allocation policies don't allow it to be used to its potential.

"Australia now has 10 gigabit links to the United States. That is as good as the Abilene network which provides high-speed connections for US universities," Foster said.

He said development work being done in New Zealand on Next Generation Internet proposals showed recognition of what was needed.

"Distributed collaboration makes most sense when you are pulling together unique expertise from different locations. In New Zealand, certainly in the sciences, there could be unique human capital, and the fact those people are in the southern oceans will not matter."

Large technology companies such as IBM, Sun and Oracle have picked up on grid computing Foster and his colleagues are developing in academia, but there are significant differences. "I think Oracle and Sun are doing some very nice work modifying their software to work on loose clusters, but I would not term that grid computing."

Much of the debate about the definition comes down to how centralised a grid should be.

Foster believes commercial computing models will focus on big centralised clusters running Linux, until web services technology is developed to tie together computers into grids.

Foster is also co-leader of the Globus Project, which is developing standards for grid computing.

Ian Foster website

The Globus Alliance