We have a lovely, lady vicar (a bit scatty at times). I don't have a problem with that - seems that it is ladies who are keeping the churches around here open. We are part of a combined Benefice so there are 2 other male vicars who appear every so often. Would attending services held by them help you? I don't know where you are, but few churches have only 1 vicar devoted to them these days.
You could attend services and not go up for Communion- you would still have the refreshment of the Liturgy.
Ecumenism isn't everyone's cup of tea - it's not mine - but do you have to be involved with that aspect? If all else fails, since the church is next door, you can at least go in there and hold your own, quiet vigil.
The Church is open for you, as it is for everyone. You could speak to the lady vic. ( who will not be offended) and she will do her best to direct you to a service which suits you. :)