"This is a really oddball galaxy," said Karl Gebhardt of the University of Texas at Austin in a press release.
"It's almost all black hole. This could be the first object in a new class of galaxy black hole systems."
The findings have been published in the journal Nature.
NGC 1277 is already the second biggest black hole ever observed, and it is a strong contender for the top spot, for the current record holder, spotted in 2011, has still not been precisely calculated. It is somewhere between six and 37 billion solar masses.
Black holes are the most powerful known forces in the Universe, creating a gravitational field that is so strong that even light cannot escape from it.
A black hole of stellar mass is formed when a very big star collapses in on itself at the end of its life.
It may then grow by gobbling up other stars and merging with other black holes, sometimes creating "supermassive" black holes which scientists say inhabit the centres of galaxies.
NGC 1277 challenges part of the galactic black hole theory because of its size relative to its galaxy.
In addition, it sits at the centre of a small disc-shaped galaxy, whereas a black hole of this size would have been expected in a far bigger blob-like, or "elliptical," galaxy.
Further work is needed to confirm whether NGC 1277 is a one-off or part of a hitherto-overlooked process of black hole creation.
"The galaxy hosting the new black hole appears to have formed more than eight billion years ago, and does not appear to have changed much since then," the Max Planck Institute said.
"Whatever created this giant black hole must have happened a long time ago."