The North has conducted three nuclear tests in 2006, 2009 and 2013 - all at the Punggye-ri site in the country's northeast.
The institute's imagery analysis would appear to confirm South Korean reports in late October that a fourth tunnel was being excavated.
In September, North Korea confirmed the restart of a nuclear reactor seen as its main source of weapons-grade plutonium.
North Korea mothballed the Yongbyon reactor in 2007 under a six-nation aid-for-disarmament accord, but began renovating it after its last nuclear test in 2013.
When fully operational, the reactor is capable of producing around 6kg of plutonium a year - enough for one nuclear bomb, experts say.
Meanwhile, North Korean leader Kim Jong Un's aunt and her husband, who acted as guardians when he was a teenager, filed a defamation lawsuit in South Korea on Wednesday against three defectors, seeking 60 million won ( $77,355) in damages, her lawyer said.
The lawsuit is unusual for North Korea's ruling family, whose members outside the country tend to shun the spotlight.
Ko Yong Suk, Kim Jong Un's aunt who defected to the United States in 1998, is accusing defectors of spreading false information that she had the leader's half-brother expelled from North Korea and that she had plastic surgery to hide after defecting, her lawyer said.
"These defectors who often make appearances on TV are not in a position to know about her directly and what they are saying is not true," Kang Yong Seok, their attorney, told Reuters by phone, declining to provide a copy of the filing.
"She and her husband find it very unpleasant." Many North Korean defectors in the South make regular media appearances.
Sometimes, the veracity of their testimony comes into question given the risk of embellishment and the difficulty of checking information they provide.
One of the defendants, An Chan Il, who fled to South Korea in 1979 and now heads a private think tank on North Korea, told Reuters he had merely repeated what had been reported in media, and said he and one of the other defendants planned to file a counter-suit for libel.
Before the couple defected to the United States, Ko Yong Suk and her husband took care of a teenaged Kim Jong Un and his younger sister, Kim Yo Jong, while they studied in Switzerland, the lawyer said.
Kim Yo Jong now holds a senior position in North Korea's ruling Worker's Party. The lawyer said Ko Yong Suk was not expected to appear in court. -AFP, AAP