While Col might not be completely wrong, before you go for the 'Instant
Karma' of the axe, try doing nothing!
Being fed\pruned so much could be the problem. Citrus don't like changing
conditions all that much, and as most of their ancestors come from tough
climates, they often shine \ flower in rough seasons. Prune lightly if at
all. If a tree has been dry for a while, a change in conditions often
provokes a reaction, ie fruit drop. And if there is no obvious signs of
hunger such as an overall yellowing of the leaves, don't feed. Also, while
you have kept the grass from around it's base, don't 'chip' the weeds
either. Don't disturb the soil around the roots, so plant something
unagressive like allysium or let the moss grow, or just keep it mulched.
Many a citrus has been killed with kindness, they like it tough, as they say
in Saville and Valencia.