There is a slim chance you are allergic to rats' urine. I am not referring to the very serious Weils disease, but an allergic reaction some people can get which does not cause serious illness.
I suggest you wear goggle - not just sunglasses or protective glasses, but swimmer type goggles - when you are next working in gardens.
In the meantime, help the specialist. Take photos of your face at varying stages of the symptoms, and keep a diary of what you do, and where, and whether you have symptoms or not.
Unless you have been advised against it, you can take antihistimine which may or may not alleviate your symptoms.
If you haven't been given any creams or lotions for your skin, E45 at bedtime may be soothing.