Mr Ellis rejected the idea he was a "nemesis" to the families.
He said they had been patient during the 10-month wait for the bodies of their loved ones to be recovered.
But he claimed a reclamation plan he devised, which involves creating a remote seal inside the mine, would be safer than the reconnaissance walk supported by mines rescue.
He would not sign off a plan with inherent risks when a safer one was available.
"What if somebody falls over? What if somebody breaks their mask? What if within 12 seconds they're a cabbage?"
The new plan would delay reentry into the mine by a month, Mr Ellis said.
He was confident the mine shaft would still be reclaimed by Christmas.
Mr Raymond said that was not good enough for families.
"Mr Ellis, another month goes by before this remote seal is done but I'm sure you'll appreciate that for a family with a man down that mine, each day is agony."
Mr Ellis accepted it was hard for families to wait for the recovery. "It must be very difficult."
Mine boss changed his story
Earlier, Mr Ellis revealed he met disaster survivor Daniel Rockhouse at his home to tell him 29 men had been killed almost instantly by the first explosion in the West Coast coal mine.
But he later gave evidence to a Royal Commission on Inquiry asserting his belief some of the men had survived the November 19 blast.
Under questioning from Mr Raymond at the inquiry this morning, he explained the details of a meeting with Mr Rockhouse shortly after the first explosion.
The meeting was requested by mine safety manger Neville Rockhouse, because he was concerned his son was suffering mentally and emotionally and blaming himself for the explosion.
Mr Ellis took Mr Rockhouse onto a porch at his home and consulted a New South Wales mines rescue manual as he told him all the men inside the mine would have been knocked unconscious by the force of the November 19 blast.
All of them - including his 21-year-old brother Ben Rockhouse - would have died soon after, he said.
That ran counter to evidence to the Royal Commission where he asserted his belief some of the men had survived the initial blast and could have taken refuge near a compressed airline in the mine.
Mr Ellis admitted there was a contradiction between his evidence and what he had told Mr Rockhouse, but defended his actions.
"I thought the best way to help that lad would be to say "it's not your fault, son".
"I tried to help the boy. He was feeling immense guilt. It doesn't alter my view that I believe there would be survivors."
Bringing up the meeting with Mr Rockhouse, was "wrong", Mr Ellis said.
He said he had genuinely held out hope that some of the 29 trapped miners may have been sheltered by mining equipment or because they were working in a dead end tunnel in the mine.
The fact he was so emotional when all hope for survivors was ruled out by a massive second blast in the mine was testament to that, he said.