You could improve the drainage of the area they appear in if it is particularly damp. Or if they are growing on old wood, remove it. But they are part of your wee garden ecosystem, maye you could learn to love them?

Besides, eveyone knows they attract fairies. Yum!