Pompeo said he had a "great visit" and a "very successful morning," adding that Trump sent his regards. Both men spoke through translators.
The secretary of state's last trip to North Korea, in July, did not go so well. He came away from it saying the two sides had made progress, only for North Korea to denounce him for making "gangster-like" demands and raising "cancerous" issues. On that occasion, he did not meet Kim.
Pompeo then planned to return in late August, only for Trump to cancel the trip at the last minute as it became apparent that the two sides remained far apart on their approach to the negotiations.
But a summit of the leaders of North and South Korea last month has helped to rekindle the peace process, as has the apparent desire of both Kim and Trump to meet again.
An official accompanying the delegation who declined to be named said the trip had gone "better than the last time" but added that it is going to be a "long haul," according to a pool report from the lone US journalist who accompanied Pompeo to Pyongyang.
Later, Pompeo flew to Seoul, where he met South Korean President Moon Jae In. He said he had "a good, productive conversation" with Kim.
"As President Trump said, there are many steps along the way and we took one of them today. It was another step forward. So this is, I think, a good outcome for all of us," Pompeo said.
Moon said he hoped a Trump-Kim summit would happen soon, and would "make irreversible, decisive progress in terms of denuclearisation as well as the peace process."
During his private meeting with Moon, Pompeo said, he and Kim had agreed to arrange the second US-North Korea summit "at the earliest possible date," according to a statement issued by Moon's chief press secretary, Yoon Young Chan.
"Secretary Pompeo said there had been discussions on denuclearisation measures to be taken by North Korea and monitoring by the US Government, as well as on corresponding measures to be taken by the United States," Yoon said.
Yoon said the two sides would form "working-level negotiating teams" to discuss the specific date and location for the summit, as well as North Korea's denuclearisation process.
At the heart of the impasse has been two fundamentally different approaches to denuclearisation.
Pompeo has insisted that sanctions on North Korea should remain in place until the country completely dismantles its nuclear programme.
The governments of both North and South Korea, however, say that is unrealistic. Instead, they want both sides to take a "phased" approach, where Pyongyang is rewarded as it takes gradual steps to roll back its nuclear programme.
On his way to Asia, Pompeo stuck to his guns, indicating that the spirit of the agreement between Trump and Kim at Singapore in June was that "we will get to denuclearisation in a fully verified, irreversible way, and then we will actually deliver on the commitments to make this brighter future for the North Korean people."
Pompeo met Japanese Prime Minister Shinzo Abe in Tokyo, and the two men "agreed that pressure must continue until the DPRK (Democratic People's Republic of Korea) denuclearizes," State Department spokeswoman Heather Nauert said.
That is not how Pyongyang sees things, nor is it the approach being advocated by Seoul.
After a summit between the leaders of the two Koreas last month, Kim said he was prepared to permanently dismantle his country's main nuclear site at Yongbyon, but only if the United States took "corresponding steps" to build trust.
North Korea has been asking for the United States to formally declare that the 1950-1953 Korean War is formally over, as a way to bring an end to hostile relations between the two countries. The war concluded with an armistice but no peace treaty.
In the past few days, North Korea has also renewed its demands for sanctions to be eased.
North Korean Foreign Minister Ri Yong Ho told the United Nations General Assembly late last month that "coercive" sanctions were lethal to trust-building between the two nations and that without trust, "there is no way we will unilaterally disarm ourselves first."
Pompeo will also visit Beijing today as he concludes his four-nation visit to the region.