Secret lives of our PCs

The discovery that computers maintain hidden files has prompted plenty of feedback, writes MICHAEL FOREMAN.

Hackers, worms, and viruses have already made people cautious when using the internet, but the existence of hidden files in Windows has made some people wonder whether it is safe to switch their computer on at all.

Since our story on the Windows spy files first appeared last week we have been inundated with hundreds of e-mails from readers. You've told us tales about e-mail eavesdropping, prying on your workmates' surfing habits, and you have reported some monstrous index.dat files.

But most of all we've received requests for further information. Has the age of Big Brother finally arrived? Here, we attempt to answer your concerns.

What is index.dat?

Index.dat is a data file that is stored by Windows in the Windows/Temporary Internet Files/Content.IE5 folder. Don't confuse it with other identically named files stored in other folders - this one behaves differently.

What does it do?

Index.dat keeps track of web addresses that are accessed by a Windows computer. Unlike similar files, index.dat in Content. IE5 is not cleared by Windows' Clear Temporary Internet command. Unless it is deleted by other means it will maintain a growing record of websites visited starting from the time Windows was first installed.

What is its purpose?

Microsoft says it is to speed surfing by keeping an index of recently visited internet addresses. But the company has not explained why the file keeps addresses that were visited a long time ago, or why no easy means has been provided to allow users to delete the file. These characteristics have led to the index.dat file being widely used for forensic purposes to gather evidence in cases involving inappropriate internet use.

What does Microsoft say?

Microsoft New Zealand technical marketing manager Terry Allen says there is nothing sinister about index.dat and its purpose was not to spy on PC users. "It is a hidden system file that optimises the performance of the internet experience," he says. The reason index.dat recorded stored web addresses was because it was "an efficient process for managing the cache", a feature of Internet Explorer that stores web pages on a user's PC to speed access to websites.

Why does index.dat keep growing?

This is how an index file works, says Mr Allen. "When files are deleted, they aren't removed, but are overwritten over time. Some [index.dat files] grow, some will not depending on how the algorithm is progressed."

Why does the file not go away even when it is deleted?

Microsoft's cache system requires the index.dat file to be present. If the index.dat file is deleted it is replaced when Windows is restarted. This file is quite small at first, around 32 kilobytes, but it grows with further web activity.

Does index.dat store web content?

No, it just stores web addresses. Content could easily be recreated by revisiting websites, but web pages may have been removed or changed since the original visit.

What about reading old web-based e-mails?

No e-mail messages are stored in index.dat, but readily browsable copies of pages from web-based mail services are stored elsewhere in the Content. IE5 folder. These may include address book pages as well as messages. Many readers have been able to exploit this characteristic to read other people's e-mails without requiring passwords, an observation which Microsoft has yet to explain. Interestingly, not every message originally accessed is stored, but we are not clear why. A large, apparently random, selection of other web content may also be stored in the folder. Microsoft recommends users who sign into Hotmail from public places or shared computers should use an increased security option that expires pages from Internet Explorer's cache when they sign out.

I've followed your Dos instructions but I can't find/edit the file - why?

If you're not familiar with Dos, it can be very tricky - one wrong keystroke or space missed and it won't work. Some readers have told us it's easier to access the file in Windows - using copy and paste to read it in Word or Wordpad.

Why can't most Windows programs find the index.dat file?

We understand Microsoft has taken certain technical steps to hide the index.dat file and other files in the Content. IE5 folder from Windows programs. This "cloaking" mechanism is loaded when Windows is first installed and may be reinforced by files in Microsoft software releases. There's probably nothing sinister about this - we believe Microsoft is doing this to protect the integrity of the system and the security of the user.

Can the file be accessed over the internet?

Contrary to a Microsoft press release, we never suggested the index.dat file was being accessed over the web. On the wider question of whether it could be accessed over the internet, the answer is no - unless your computer has been compromised by a hacker.

Is Microsoft accessing the file?

When we've asked this question, senior Microsoft New Zealand staff have consistently and repeatedly denied it. "It is not designed to be accessed by Microsoft or any third party. It's not designed for people to look at," says Mr Allen.

Why are you targeting Microsoft? Don't other browsers keep similar files?

Yes they do, but they are cleared by Windows commands. The file persistence problem is caused by the special interaction of Internet Explorer with Windows. One long-term solution could be to use a non-Microsoft browser such as Netscape or Opera with Windows.

Is this old news?

Yes and no. The topic has been bubbling away on net discussion groups for years, and plenty of technically savvy people already knew about these files. But some were unaware of its properties or the other contents of the Content. IE5 folder.

Judging by your e-mails, the existence of index.dat was certainly news to most people - and a bit of an eye opener.

* Michael Foreman would like to acknowledge the contribution of a white hat hacker called "The Riddler" whose postings have assisted our research.