"The challenge over the last couple of months is that we are in full production," Houston said.
"We have quadrupled the processing power, tripled the storage and added another two layers of complexity to the network while the facility is running at warp factor 10.
"It's like changing the engines on a 747 while we're at 20,000 feet flying at mach 2 with 300 passengers on board."
When New Line executives visited three weeks ago to check on progress, they approved Houston's request for "more horsepower" - an increase in the size of the render wall, the racks of processors which add texture and shading to the animations.
Over the past week 476 servers, each with two 2.2GHz Xeon processors and 4 gigabytes of memory per processor, have been added to the wall, supplementing the 350 existing 1GHz Pentium 3 servers.
The servers came from Renaissance subsidiary Insite Technology and were put together by systems integrator Digital Video Technologies as fast as Intel could supply the chips.
A box room was turned into new server room to house them, and members of Houston's 30-strong IT team laboured around the clock to bring up each server rack, plugging them in to what switch provider Foundry Networks says is already the largest TCP/IP network in the Southern Hemisphere.
As each rack is brought up it is assigned a name. The landmarks from Middle Earth - Amonsul, Baradour, Carndun, Durthang, Minas Tirith and so on - have run out, and names like Ruapehu and Tongariro are being pressed into service.
The servers run Pixar's Renderman software on Red Hat Linux. Houston said Weta had effectively moved to Linux on the desktop, with the artists using the Maya animation suite from Alias/Wavefront and the Shake compositing tool from Nothing Real, which was now owned by Apple.
Other applications used include Houdini, Softimage 3D and Commotion.
There are also internally designed plug-ins and applications such as Massive, which is used to create crowd scenes full of creatures behaving in a realistic manner.
"There are people who work on the look of each creature, there are people who work on the animation, how it walks and moves, there are people who work on lighting and shading and textures, there is an environments area which might add smoke or cloud or fire or water, all those layers add to the shot and get composited into the final image," Houston said.
The final compositing is still done on an eight-processor SGI Onyx because the application used, Inferno from Discrete Logic, runs on SGI's Irix operating system.
Artists' primary workstations are new IBM dual-processor machines, with most keeping hold of their old SGI machines for checking certain effects.
"There were early issues about Linux stability and driver support. Most of those have been resolved. So it's simple bang for buck," Houston said.
When artists leave their workstations, monitors on the network detect the decline in activity and add the processing power to the render wall, which will eventually get up to 1300 processors.
More than two terabytes of data is being produced at Weta every day, but each render doesn't need to be stored - only the scripts which specify which elements will be included in a shot.
That means only about 300 gigabytes of data needs to be backed each night.
Houston has about 200 terabytes of storage to keep track off. He has created a separate room, freeing space in the heavily air-conditioned original computer room for more server racks.
In pride of place is a 60 terabyte StorageTek L700 robotic storage library, which replaces a 20 terabyte StorageTek 9710 library.
A 12-processor SGI Origin 2000, running SGI's Data Management Facility (DMF) software is used to manage the movement of data from online to near line storage.
This allows artists to access previously used images when needed without tying up more expensive online storage.
"What we need to do is give the artists as much processing power and storage as possible," Houston said.
Feature: Lord of the Rings
Special LOTR report: A long expected party
Best Lord of the Rings websites
