But despite the threats, it was the women who had to end up being moved.
Mr Rotem said that eventually, "after a lot of writhing, shouts and manoeuvring", an elderly American woman and a young Israeli woman agreed to switch seats, allowing the flight to finally takeoff.
Mr Rotem described one of men as "particularly devout and ascetic" having even got on the plane with his eyes closed and keeping them shut for the duration of the flight in an effort to avoid looking at any woman on board.
But, he noted, other Orthodox Jewish men aboard the flight expressed "surprise and revulsion" at the four ultra-Orthodox men's conduct.
In 2016, a similar case of discrimination was filed against the airline by a female passenger. Israel's Supreme Court then ruled that asking a passenger to move their seat based on gender is a form of discrimination.
In a statement, El Al said discriminating against airline passengers is "forbidden" and that it does its utmost to serve a "wide array of populations and travellers."
The Israel Religious Action Center, a progressive group that led last year's lawsuit, accused the airline of breaking its commitments.