You might as well ask why you don't see white British doing manual work! Well,you don't in this area if you visit the vegetable growers. The hard work is all being done by Eastern Europeans because the native locals don't want to do it. The same applies to a large degree in other industries such as the hotel trade.
Why were so many "Asians", all Muslims, brought into Northern towns such as Bradford? Was it not because they were wanted to work in the textile industry? And was that because locals did not want to do the arduous work? Whatever the reason, the immigrant labour did that work.
There are plenty of Muslims still doing manual labour. The OP must be living in an area with few Muslims.
Rugby is not parr of the culture of the country they or their ancestors came from. Frankly, rugby union isn't part of the "culture" of most non-Muslims here. It is if you went to a rugby playing school as e.g. in Wales, where it is an important part of sporting life.