With the benefit of a nice warm location for their nest, they may well not die off until well into winter.

I successfully got rid my first cavity wasps' nest simply by blocking off the access to it. I think I got that one quite early. It didn't work the second time because they found other ways out that were less congenial to me, so I made a strategic decision to reopen the original entrance.

Wasps chew wood to make their nests. The second cavity wasps' nest I had was just below a wooden windowsill, and I could hear them chewing at the windowsill, it was quite loud.

There are aerosol wasp nest killers that are very effective and easy for the kind of wasps nest that is confined inside a hole such as a cavity wall. The trick is to use it at night (that means well after dark) when the wasps are mainly dormant. That way you aren't interfered with, and you get most of them as they are all at home. I have used them a couple of times with success.

When the wasps start losing their way and regularly coming into the part of the house that you occupy, as they will, you may wish you'd got rid of it sooner. You can end up with dopey wasps crawling over your bedding in the early morning, (he says from experience) which sets yourself up perfectly for the sting.