Not FIFA's fault if a road collapses or a workman dies while building one of the stadiums.
It could happen in any country, and there were problems in South Africa before the last World Cup.
And I found this on the internet about the Olympics:
"2 fatalities during the 1996 Barcelona Olympics construction, one fatality during the 2000 Sydney Olympics construction, 14 killed during the 2004 Greece Olympics construction and 10 killed during construction for the 2008 Beijing Olympics"
p.s. To put things in perspective it is estimated about 30,000 people died building the Panama Canal.