'Hang it all' does not have a vulgar or offensive meaning, as far as I know, and nor does it mean 'stop doing it'. It is just a euphemistic or more polite way of saying 'damn it all'. 'Hang on' means 'stop'.
The 'it' is the same 'it' as we find in phrases such as 'it's raining'.
Overall, the phrase suggests that the speaker has had enough of whatever the annoying situation/matter is and is just giving up on bothering about it any more.