If you remember the comedian, Tommy Cooper, you'll know what a fez is. It's the hat - generally reddish in colour - that he always wore in his act. It was originally associated with the city of Fez in Morocco, where such hats were made, and it became the national headgear of the Turks.
Organ-grinders - men who busked using a mechanical organ to create 'music' - often had a monkey as a companion and to attract punters. The monkey generally wore such a fez.
All I can suggest as to the reason is that such street-entertainers may have started in the Mediterranean region where the fez was commonly worn. Apart from that, we may as well ask why Tommy Cooper wore one, too!