Until around 1600, the regular literary form of the verb meaning �ask' was 'ax', based on earlier forms 'acsian' or 'axian'. 'Ax' remained until then the commonest form throughout the Midlands and Southern England, whilst 'ask' was essentially just the Northern form of the word.
It is, therefore, not just confined to specific racial groups but - even now - largely dialectal. I believe 'aks' is still quite commonly used in Lancashire, for example.
So, in a way, the people who use aks/arks are the ones who stuck with the original. Perhaps they should ask why you say ask!