I don’t plant to go there again however man cannot say like that. Allah (SWT) knows the best what will happen.
I have visited different Holy Places. I was at St. Peter's Basilica in Vatican what is the number 1 Sanctuary of the Roman Catholic Church. I went to see the wailing wall in Israel and entered the Church of the Holy Sepulchre. I could feel the spiritual atmosphere of these Sanctuaries and the same I felt in Mecca. However, nobody said me in Vatican and Jerusalem that these places are only for Christians and Jews so I have to turn back. There is nothing in the Quran what prevents believers of the One God out of Muslims to enter these sites. According to Quran everybody is Muslim who follows the faith of Abraham.
There is a caption above the motor way to Mecca. “Muslims Only” is written above the lanes what lead to Mecca and For Non-Muslims for those who are not Muslims. This latter one turns back. The reason of this regulation goes back to the following Quran aya:

O ye who believe! Truly the Pagans are unclean; so let them not, after this year of theirs, approach the Sacred Mosque. And if ye fear poverty, soon will Allah enrich you, if He wills, out of His bounty, for Allah is All-knowing, All-wise. (Quran 9:28)

In the Arabic text we read “mushrikun” as Pagans. Mushrik is the one who associates partners to God. They are unclean and they have no place in the Sanctuary. We are speaking about physical and spiritual purity at the moment. However, who are those who associate partners to God? They are those who elevate man, philosophy or thought to the level of God. Such are the dictators or principles what prevail over divine revelations or man who expose themselves as unerring etc.
Christians are blamed as “mushrik” because they consider Jesus as son of God. I think we have to pass some steps beyond this Medieval stupidity. Faith is One! On the other hand, I have some idea who cannot enter to Mecca among those who are in Saudi Arabia. And they are not the ones in the faith of One God!

I have another comment, too. Holy Sanctuaries of Mecca and Medina are “capitals” of all Muslims. If authorities declare exclusivity for Muslims in these cities why have I apply for visa in order to enter?
At the moment Mecca and Medina are part of a Kingdom and it is normal to apply for visa to the Kingdom. However, it is not logic to declare: Muslims Only! It is a clear statement that Mecca and Medina doesn’t exist under Muslim rule.