It is understood he was at the pools with members of his family and friends.
"It was a stressful and upsetting time for the friends at the scene and police were called to assist," a police spokeswoman said.
"The matter is being referred to the Coroner. WorkSafe have also been notified."
The man's name is yet to be released.
A woman who was at the baths at the time spoke to the Herald this morning about the tragedy.
"I witnessed the event, it was an extremely traumatic and sad event for not only everyone at the pools today but also his family and friends who were there," she said.
The woman, who asked not to be named, was a the baths with a friend when the man was pulled from the water.
"We were laughing away having a good time and then I saw this big guy pull another big guy out of the water, he pushed him onto the side and started doing CPR.
"I was like 'oh my God', I was in shock."
The lifeguards raced over to the man giving CPR to the victim, but let him carry on.
The woman said he did CPR for about 20 minutes before paramedics arrived and took over.
She did not think the men knew each other.
"He was just someone at the pool - he was amazing, I was really proud of him and I hope he gets some recognition for what he did."
While CPR was going on everyone else at the pool was asked to leave.
The woman waited nearby though, hoping to find out that the man had been revived.
Sadly, that did not happen.
"After a long time they stopped doing CPR, then I heard crying," she said.
The dead man was surrounded by a group of people that she thought were friends or family.
"Then I knew it was over," she said.
"It was really awful... I was in shock and I couldn't sleep all night thinking about it all."
She said her thoughts were with the man's family.
Parnell Baths is run by Auckland Council, who released a statement about the man's death this morning.
"Auckland Council is incredibly saddened following the death of a young man at the Parnell Baths last night and our thoughts are very much with the man's family and friends at this time," said the council's head of active recreation Rob McGee.
"It is believed the man suffered a medical event while swimming and was pulled from the water unconscious.
"Our thoughts are also with those staff who tried to revive the man, as well as other pool users who were there at the time, and support will be offered to them in the coming days."
McGee as a mark of respect for the man and his family, the Parnell Baths will not open at its usual opening time this morning.