But there were many flaws in Morsi's brief rule which he himself admitted. The economy was going nowhere. Shortages of basic utilities continued. On the international stage, Morsi often appeared more like a cuddly grandpa than a statesman. He was the subject of jokes, and Egyptian TV satirists had a field day. You know you're heading toward democracy when things are bad but people are free to ridicule their leaders.
Before the army removed Morsi's government and took him and senior members of his party into custody, Western news channels showed the image of a full Tahrir Square calling for his resignation. Al-Jazeera English showed images of Tahrir Square and other parts of Cairo side by side, one half of the screen showing millions supporting the President and the other showing those opposed to him. Prayers and women donning hijab could be seen in both camps. This was about politics, not religion.
Perhaps the best summary of the situation in Tahrir Square when the military coup was announced last week was from UK Channel 4's roving reporter Krishnan Guru-Murti, who was on the scene. As the fireworks went off, he announced: "This is the celebration of the end of democracy." One jubilant protester he interviewed told him: "We feel excited. We feel so happy. We don't believe it. We don't believe it. We can do this every time we have a president that who ignore us, that who doesn't see us, we will not just throw him away but we will kill him."
In many Western circles, the Brotherhood are seem as nasty, beady-eyed ayatollahs who could never work with "us" (or at least with the US). But Egypt's history tells a different story.
In his book Nasser The Last Arab, Palestinian journalist Said K. Aburish writes about Egyptian dictator Gamal Abdel Nasser's paranoia about the Brotherhood. Aburish, who has no great love for the Brotherhood, further alleges that they received support from the United States, who prevailed upon Saudi Arabia to bankroll the Brotherhood not just in Egypt but across the Arab world. Aburish cites an interview with the son of Said Ramadan, one of the Brotherhood's senior leaders, who confirmed this. Jordan's King Hussein provided Brotherhood leaders with diplomatic passports, and millions were transferred into Ramadan's Swiss bank account.
Despite being officially banned, the Brotherhood continued their involvement in civil society. They infiltrated and took over key professional bodies and educational institutions. They also provided key social services - hospitals, clinics, schools, welfare offices - which successive governments were unable (or too corrupt) to provide. Allegedly, secular political leaders were happy for the Brotherhood to play this role as it freed up government funds for more useful pursuits such as graft and corruption.
So whether the Brotherhood runs in any future election under their own banner or under the banner of allies in the trade union movement, the fact remains that they have over the decades gained enormous support and goodwill at a grassroots level.
And they must be allowed to participate. Complete suppression of the Brotherhood will help no one. If anything, there is a risk that Brotherhood branches in other parts of the Arab world may regard legitimate democratic processes as being a waste of time.
The West needs to pressure the Egyptian interim government to allow free and fair elections for all. The military must be warned to return to the barracks, not rewarded with extra aid as the US seems to be doing. And ordinary Egyptians need to learn that calling in the army will only make things worse.
Irfan Yusuf is an Australian lawyer and author.