Imaam Abu Muhammad Al-Hasan b. `Alee b. Khalaf al-Barbahaaree (died 329H, rahimahullaah) stated in his Sharhus-Sunnah (point 37):
ولا بأس بالصلاة في السراويل
“And there is no harm in praying in loose trousers.”
As-Saraaweel (trousers) is singular, and it is something well-known. It is worn over the `awrah (parts of the body that must be concealed). And it is a garment that is stitched in proportion to the lower half of the body.
He said: The prayer is correct and sound whilst wearing loose trousers. And this applies to the men, because the `awrah of a man is from the navel to the knees – and the trousers conceal that area. So if a man was to prayer in trousers whilst covering what is between the navel and the knees, then his prayer is valid.
As for the woman, then all of her is considered `awrah in the prayer except her face as long there are no non-mahram  men present.
(*Mahram is any male that she is presently married to or any other male that she is permanently forbidden to marry.)
If a man was to prayer in an izaar (waist sheet or wide sarong) then that is better than praying in wide trousers, and praying in a long shirt (i.e. a thawb or robe) then that is better still because that is more beautiful in appearance. Allaah, the Most High, said:
يَا بَنِي آدَمَ خُذُوا زِينَتَكُمْ عِندَ كُلِّ مَسْجِدٍ
O children of Adam, take your adornment at every masjid..
Meaning here, during every prayer. And adornment is a more general term than merely covering one’s `awrah .
 In this speech of al-Imaam al-Barbahaaree, rahimahullaah, is a refutation upon some of the sects of the Shee`ah who hold that it is forbidden to pray in loose trousers because they hold that the garment is affected by wind exiting the anal passage. They, therefore, hold it obligatory to remove the trousers before performing the Salaah.
NOTE by Translator:
The intent here behind “loose-trousers” is not the very tight-fitting trousers worn in the Western world and imitated by some Muslims in their lands. These Western trousers/pants are tight around the buttocks (and even tighter when one is in rukoo’ and sujood of the prayer), and their hems hang below the ankle bone. It is not permitted to pray in these tight trousers that expose the ‘awrah of the man – indeed a Muslim should not even wear them as they imitate the particular customs and fashions of the non-Muslims, and they fall below the ankle-bones. The Muslim is permitted, however, to wear and pray in loose trousers/pants that are wide, loose and are raised above the ankles: