Firstly, unless there's a covenant relating to the upkeep of boundary fences, there's no reason that either of you is automatically obliged to repair the fence. Whoever owns it can simply leave it in a poor state of repair or remove it.
Secondly, there's no universally reliable way of establishing who owns a fence - but consider how most people would erect a fence at the front of their property. Almost everybody would erect it with the fence posts on the inside of the fence. This convention (which is not a rule) often applies to boundary fences as well. I.e. if the convention has been followed, the owner of the fence will be the person who has the fence posts on their side of the fence.
As stated, it's only a convention, not a rule. Further, it won't help much if the fence has 'in-line' posts.
Chris