Yesterday, defence lawyer Ian Brookie sought interim name suppression for his client based on fair trial rights, which was opposed by police, the Millane family and the press.
Judge Thomas declined the application for name suppression, but Brookie appealed the decision, which automatically imposes a 20 working day suppression under New Zealand law.
Little issued a stern warning, but acknowledged that British media were outside the jurisdiction of New Zealand courts.
"The international media, particularly the British media, are not helping the Millane family. If the media are concerned about justice and doing justice for the Millane family, they should stop publishing details.
"We don't have any arrangement whereby we can stop media from overseas publishing information. What we can do is act as tightly as we can here."
Yesterday, Detective Inspector Scott Beard, of Auckland City Police, reminded the public that the 26-year-old had interim name suppression.
National leader Simon Bridges, a former Crown prosecutor, said suppression orders had a place but would not go as far as asking British media to stop naming the accused.
He said the case was "very affecting".
"Having worked on cases that have some similarities to these, I know what an affect they have on the family involved. For a father to have to come out here to a foreign land very far away, it's heart-wrenching.
"All New Zealanders will feel deeply for them and be moved by this case."
Prime Minister Jacinda Ardern became emotional at her press conference yesterday as she apologised to the Millane family on behalf of New Zealand, saying that Grace Millane should have been safe in New Zealand.
Little echoed that sentiment this morning.
"When a young woman, really on the cusp of the future of her life, is snuffed out in the way Grace Millane has been, we're all going to appalled by it, ashamed that it happened in our country."
He said the Government was tackling New Zealand's issues of family violence and violence against women.
"We've got to fix that problem. It underpins a whole heap of problems that we've got.
"We've seen another victim of it, and a visitor to our shores."