The question is right if we consider this world to be an eternal abode. Here comes the question of faith, that is, what we believe in. Monotheistic religions, including Islam, believe that this world is a transitional state. Nothing here is eternal and the earthly being has only one purpose: God is testing us with all kinds of trials. The final resting place of the soul depends on the result of the test. If we do not conceive of it in this way, then the many injustices, war, vicissitudes, diseases that we must endure are truly meaningless.
I have to start with the quote that was exposed recently, sorry for the repetition:

If Allah so willed, He could make you all one people: But He leaves straying whom He pleases, and He guides whom He pleases: but ye shall certainly be called to account for all your actions. (Quran 16:93)

Allah's Will and Plan, in allowing limited free-will to man, is, not to force man's will, but to give all guidance, and leave alone those who reject that guidance, in case they should repent and come back into Grace.
We also find as previously the active and passive form of His guidance. 1. Active: He guides whom He wants. 2. Passive: He guides the one who wants Him. According to the Arabic text both are correct. I agree with the latter one.

Any philosophy that concentrates on one aspect of the creation and denies or ignores the other side is partially true and partial truths are no truth at all.

First of all, Allah did not make this world a permanent world. This is a temporary world and everything here has a time limit. When its times comes it will die, come to an end and finish. Neither the good things of this world are forever, nor the bad things eternal. We are here for a short time and we are being tested.

Allah has placed a physical law and a moral law in this universe. Allah allows suffering to occur when one or more of these laws are broken. The physical law is based on cause and effect. Sickness comes if one does not take care of one’s health or is exposed to infections. A car accident occurs when one is not alert, or drives in a careless manner, or if the cars are not checked, roads and freeways are not made and kept in right shape, or the traffic laws are not right or not properly enforced. Study of causes and effects is very important to facilitate safeguards. Even here we should keep in mind that Allah often saves us and He does not let us suffer from every negligence.

If Allah were to punish men according to what they deserve. He would not leave on the back of the (earth) a single living creature: but He gives them respite for a stated Term: when their Term expires, verily Allah has in His sight all His Servants. (Quran 35:45)

There would be no salvation for any of us if we went merely on our deserts. It is Allah's mercy that saves us and helps us to a better and better life until we attain the goal of our existence.

But sometimes Allah does punish people because of their violations of His laws whether they are physical or moral. The Quran tells us that many nations and communities were destroyed because of their sinful lifestyles (22:42-45)

Suffering can also be a test and trial for some people. Allah allows some people to suffer in order to test their patience and steadfastness. Even Allah’s Prophets and Messengers were made to suffer. Prophet Ayyub is mentioned in the Quran as a Prophet who was very patient. Good people sometimes suffer but their sufferings heal others and bring goodness to their communities. People learn lessons from their good examples. Martyrs die for their faith, soldiers give their lives for their nations and this brings liberation and freedom for their people.

Allah sometimes allows some people to suffer to test others, how they react to them. When you see a person who is sick, poor and needy, then you are tested by Allah. Allah is there with that suffering person to test your charity and your faith. In a very moving Hadith Qudsi (Divine Hadith) the Prophet, peace be upon him, said:

On the authority of Abu Hurayrah (may Allah be pleased with him), who said that the Messenger of Allah (peace be upon him) said:
Allah (mighty and sublime be He) will say on the Day of Resurrection: O son of Adam, I fell ill and you visited Me not. He will say: O Lord, and how should I visit You when You are the Lord of the worlds? He will say: Did you not know that My servant So-and-so had fallen ill and you visited him not? Did you not know that had you visited him you would have found Me with him? O son of Adam, I asked you for food and you fed Me not. He will say: O Lord, and how should I feed You when You are the Lord of the worlds? He will say: Did you not know that My servant So-and-so asked you for food and you fed him not? Did you not know that had you fed him you would surely have found that (the reward for doing so) with Me? O son of Adam, I asked you to give Me to drink and you gave Me not to drink. He will say: O Lord, how should I give You to drink when You are the Lord of the worlds? He will say: My servant So-and-so asked you to give him to drink and you gave him not to drink. Had you given him to drink you would have surely found that with Me. It was related by Muslim.

Arabic/English book reference: Hadith 18.

Prophet Jesus (peace be upon him), is also reported to have said something similar. (See: Matthew 25: 35-45)

So, to summarize, we can say that sufferings occur to teach us that we must adhere to Allah’s natural and moral laws. It is sometimes to punish those who violate Allah’s natural or moral laws. It is to test our faith in Allah and to test our commitment to human values and charity. Whenever we encounter suffering, we should ask ourselves, “Have we broken any law of Allah?” Let us study the cause of the problem and use the corrective methods. “Could it be a punishment?” Let us repent and ask forgiveness and reform our ways. “Could it be a test and trial for us?” Let us work hard to pass this test.

Believers face the sufferings with prayers, repentance and good deeds. The non-believers face the sufferings with doubts and confusions. They blame Allah or make arguments against Him.

May Allah keep us on the right path, Amen!