Me and my neighbour (Semi-detached houses) have drives and there's room for 2 cars in between our drives, on the front street. I don't mind when cars park outsie our house, except when some thoughtless div parks in the middle, taking up both parking spaces.
Anyway, you're doing nothing illegal, as long as you're not causing an obstruction/hazzard of any kind, so just ignore them if they tell you otherwise, 'cos as already stated on here, it's a public highway and as long as there are no parking restrictions in place, you can park there when you want and for as long as you want.