Not necessarily. In summer the warmth "contours" run (to a first approximation) east-west, so you get steadily cooler as you go north. But in mid winter the contours run roughly north-south, so the west of the country is warmer than the east. If your bit of the deep south is, say, Kent, then your lowest winter temperatures are going to be lower than places on the W coast which are a good deal more north - think of the famous gardens at Inverewe in Scotland, for example.

What really makes it more difficult to grow things in the north (annual things, rather than hedges) is the length of the growing season, which can be a full two months less than in the south. So it's always a balance - you can't sow stuff as early as in the south, but you have to try to sow it early because your growing season will end earlier.