A colleague of mine applied for planning permission to build a 'granny
annexe'. At that point, the planners said 'wait a minute ... there
should be a field here, not a garden ..' and he was told to reinstate
the field and the original boundary. He felt especially peeved as it had
been the previous owner who'd made the change!

The buyer or his solicitor should have found this out before he bought it.
Which is why, as a result of a recent court case, solicitors are insisting
sellers provide a declaration and/or proof that all development has been
done with permission. This applies to building regulations approval as well.
The bottom line now is that without the required approvals a property
becomes unsaleable. If the seller makes a false declaration then they are
liable for the consequences. AFAICR the case in point cost the seller some
£67,000.