Few are willing to discuss their fears openly because of the risks associated with criticising official policy in the authoritarian kingdom. And, with the exceptions of Turkey and Iran, fellow Muslim nations have largely held their tongues for fear of of a diplomatic fallout and restrictions on their citizens' pilgrimage visas.
Western archaeologists are silent out of fear that the few sites they are allowed access to will be closed to them. But a number of prominent Saudi archaeologists and historians are speaking up in the belief that the opportunity to save Saudi Arabia's remaining historical sites is closing fast.
"No one has the balls to stand up and condemn this cultural vandalism," says Dr Irfan al-Alawi who, as executive director of the Islamic Heritage Research Foundation, has fought in vain to protect his country's historical sites.
"We have already lost 400-500 sites. I just hope it's not too late to turn things around."
Sami Angawi, a renowned Saudi expert on the region's Islamic architecture, is equally concerned.
"This is an absolute contradiction to the nature of Mecca and the sacredness of the house of God," he told the Reuters news agency earlier this year.
"Both [Mecca and Medina] are historically almost finished. You do not find anything except skyscrapers."
Dr Alawi's most pressing concern is the planned £690m expansion of the Grand Mosque, the most sacred site in Islam which contains the Kaaba - the black stone cube built by Ibrahim (Abraham) that Muslims face when they pray.
Construction officially began earlier this month with the country's Justice Minister, Mohammed al-Eissa, exclaiming that the project would respect "the sacredness and glory of the location, which calls for the highest care and attention of the servants or Islam and Muslims".
The 400,000 square metre development is being built to accommodate an extra 1.2 million pilgrims each year and will turn the Grand Mosque into the largest religious structure in the world. But the Islamic Heritage Foundation has compiled a list of key historical sites that they believe are now at risk from the ongoing development of Mecca, including the old Ottoman and Abbasi sections of the Grand Mosque, the house where the Prophet Mohamed was born and the house where his paternal uncle Hamza grew up.
There is little argument that Mecca and Medina desperately need infrastructure development. Twelve million pilgrims visit the cities every year with the numbers expected to increase to 17 million by 2025.
But critics fear that the desire to expand the pilgrimage sites has allowed the authorities to ride roughshod over the area's cultural heritage. The Washington-based Gulf Institute estimates that 95 per cent of Mecca's millennium-old buildings have been demolished in the past two decades alone.
The destruction has been aided by Wahabism, the austere interpretation of Islam that has served as the kingdom's official religion ever since the al-Sauds rose to power across the Arabian Peninsula in the 19th century.
In the eyes of Wahabis, historical sites and shrines encourage "shirq" - the sin of idolatry or polytheism - and should be destroyed. When the al-Saud tribes swept through Mecca in the 1920s, the first thing they did was lay waste to cemeteries holding many of Islam's important figures. They have been destroying the country's heritage ever since. Of the three sites the Saudis have allowed the UN to designate World Heritage Sites, none are related to Islam.
Those circling the Kaaba only need to look skywards to see the latest example of the Saudi monarchy's insatiable appetite for architectural bling. At 1972ft (601m), the Royal Mecca Clock Tower, opened earlier this year, soars over the surrounding Grand Mosque, part of an enormous development of skyscrapers that will house five-star hotels for the minority of pilgrims rich enough to afford them.
To build the skyscraper city, the authorities dynamited an entire mountain and the Ottoman era Ajyad Fortress that lay on top of it.
At the other end of the Grand Mosque complex, the house of the Prophet's first wife Khadijah has been turned into a toilet block. The fate of the house he was born in is uncertain.
Also planned for demolition are the Grand Mosque's Ottoman columns which dare to contain the names of the Prophet's companions, something hardline Wahabis detest.
For ordinary Meccans living in the mainly Ottoman-era town houses that make up much of what remains of the old city, development often means the loss of their family home.
Non-Muslims cannot visit Mecca and Medina, but The Independent was able to interview a number of citizens who expressed discontent over the way their town was changing.
One young woman whose father recently had his house bulldozed described how her family was still waiting for compensation.
"There was very little warning; they just came and told him that the house had to be bulldozed," she said.
Another Meccan added: "If a prince of a member of the royal family wants to extend his palace he just does it. No one talks about it in public though. There's such a climate of fear."
Dr Alawi hopes the international community will finally begin to wake up to what is happening in the cradle of Islam.
"We would never allow someone to destroy the Pyramids, so why are we letting Islam's history disappear?"
- INDEPENDENT
