there is no such thing as your fence and his fence, there is a sort of
commonly misunderstood convention just as there's a misunderstanding
that a common law wife has some status in law.

Sorry to say it but that is complete and utter twaddle.

For a detailed look at garden law in general and fences/boundaries in
particular take a look here.

http://www.gardenlaw.co.uk/