There is a lot of logic. The banks have been hardest hit. Already it is clear London is no longer going to be the financial centre of Europe, housebuilders are less interested in building houses/flats as mortgages are going to be harder to come by and foreign investors are pulling out (may be good news in long run), and firms are finding contracts from Europe are being put on hold.
And it's not rich bankers doing it- our pension funds are doing it too as they look after our money. It's my life policies and shares ISA. And why would anyone institution hold sterling when it wants more certainty.
But we can't turn the clock back and will have to have confidence in our politicians to deliver a good long term outcome