"If [the doctor's] name is published, can the public join the dots and come out with identity of Ms X? We say yes," he said.
He said the pair shared a recreational interest that saw them spend significant time together.
Though the relationship was secret, the doctor said people quipped: "You're a bit old for her, aren't you?"
McLelland also said suppression would benefit his client's wife and family.
But the court heard the patient was "strongly opposed" to the man getting name suppression.
Counsel for the committee Kate Feltham argued cancellation was the only appropriate penalty, given the tribunal highlighted 15 serious aggravating features behind the breach of professional standards.
She said there was a "well-established" doctor-patient relationship, which began when the woman was a teenager.
While their sexual bond was only formed many years later, the context was "hugely important".
Feltham said the power imbalance of the doctor-patient relationship was exacerbated by a significant age gap.
The court heard the woman discussed personal matters with the doctor, who had provided emotional support and performed examinations of a sensitive nature over a lengthy period.
While the patient transferred to a new GP at the same practice when their relationship became physical, her next five appointments were with her original doctor, Feltham stressed.
"[He] should have referred her to another practice altogether."
McLelland suggested the appropriate penalty imposed by the tribunal should have been suspension and a fine.
"It's just one patient, he's not a predator or anything like that," he said.
He stressed the relationship had grown through a hobby rather than purely in a clinical setting.
McLelland was also critical of a victim impact statement provided to the tribunal at the eleventh hour.
He had no opportunity to cross-examine the patient nor was it clear how much weight the statement was afforded, he said.
Feltham, however, argued the significance of the document was "wildly overstated".
Its contents were "pretty benign", she said, and it would not have swayed the decision-makers to any great degree.
Justice Robert Osborne reserved his judgment.