The scented pelargoniums are good as is lavender sage , many of the
sages are scented my best one being blackcurrant sage.
Pheasant berry is brilly but it does not smell and gets rather large
altho it does get cut back every year, how about Mahonia for smell

But the scented pelargoniums aren't hardy, surely?

Perhaps I have been lucky so far, they are in a very sheltered position.