Pompeo said in his new book, Never Give An Inch: Fighting for the America I Love, he does "not think the world properly knows just how close the India-Pakistan rivalry came to spilling over into a nuclear conflagration in February 2019", the BBC reports.
"The truth is, I don't know precisely the answer either; I just know it was too close," he wrote.
Sign up here to receive our daily newsletters and breaking news alerts, sent straight to your inbox.
Relations between India and Pakistan have been often strained over decades but in early 2019 they hit a new freezing point after Delhi launched strikes against militants in Pakistani territory following an attack on Indian troops in Kashmir.
Pakistan's military then announced it had shot down two Indian air force jets and captured a pilot.
Pompeo said he'll always recall the night he was in Vietnam at a summit "when "India and Pakistan started threatening each other in connection with the decades-long dispute over the northern border region of Kashmir".
The former top US diplomat said he was awakened in Hanoi to speak with an Indian "counterpart", who is unnamed.
"He believed the Pakistanis had begun to prepare their nuclear weapons for a strike. India, he informed me, was contemplating its own escalation," Mr Pompeo writes.
"I asked him to do nothing and give us a minute to sort things out."
In his book, Pompeo how he and then US national security adviser John Bolton began to try and diffuse the crisis.
Pompeo contacted Pakistan's then army chief General Qamar Javed Bajwa, with "whom I had engaged many times", and told him what the "Indians had told me".
"He said it wasn't true. As one might expect, he believed the Indians were preparing their nuclear weapons for deployment. It took us a few hours – and remarkably good work by our teams on the ground in New Delhi and Islamabad – to convince each side that the other was not preparing for nuclear war.
"No other nation would have done what we did that night to avoid a horrible outcome," Mr Pompeo said.
Then Australian foreign minister Marise Payne in February 2019 was among leading diplomats to urge a calming of tensions between India and Pakistan. She warned how further military action in Kahsmir could trigger a far worse global conflict.
Pakistan and India have fought three wars since independence from British colonial rule in 1947, two over the disputed Himalayan region of Kashmir, and went to the brink a fourth in 2002 after a Pakistani militant attack on India's parliament.