That was in 2001, when Federer was ranked 29 in the world and lost 6-7(5) 4-6 4-6 to world No 18 Arnaud Clement in the third round.
Since then Federer has been a permanent fixture in the closing stages, his sheer presence elevating the event.
Early last week, while watching a match on an outside court, I noticed this growing, discernible buzz coming from the opposite court. "That's Fed," said a security guard, "he's on his way."
A few minutes later he appeared, and hundreds ringed the court to watch him train, while Swiss fans displayed 'Quiet please, genius at work' signs.
Federer attracts the biggest crowds to Rod Laver Arena, no matter who he is playing or at what time, because there is no one else like him. At a time when men's tennis is dominated by power and can be predictable, Federer can come up with shots that make you sit up in your seat. His appreciation of angles and touch are unique in the modern game and a simple flick of the wrist can leave his opponent - and spectators - gasping in amazement.
Friday's match was incredibly strange. The crowd initially cheered the underdog, Andreas Seppi, in the opening sets, perhaps grateful to be getting a contest as the Italian had taken only one set in their 10 previous encounters. But after the second set tie break (won 7-5 by the Italian) you could sense the concern; what was wrong with Roger? The Swiss looked hesitant, uncertain and unsure, as the stadium willed a comeback. There was relief when Federer won the third set, and disbelief when Seppi got the winner on match point - "the shot from I don't know where" - as Rafa Nadal would later describe it.
There were huge cheers for the Italian, a sustained ovation for the departing Federer and then stunned silence.
Later, as around 100 journalists from across the world waited in the press conference room, no one was talking. There was an eerie silence around the usually chatty scribes. A downcast Federer eventually appeared, wearing a shirt emblazoned with TNNS across the front (like his game that day, missing the crucial elements) and answered questions in English, Swiss-German, French and German. The usual jokes, the banter, were all gone as he struggled to explain the unexplainable, why he "was on the plane and [Seppi's] not".
And then Federer was gone, out a side door.