The message
from Kay contains these words:

It's not the pollen per se, it's the particular dye that lily pollen
has. Lilies are known for their ability to stain anything in sight.

The best way I've found to deal with lily pollen on clothes is not to touch
it before lifting it off with Sellotape. Sellotape can lift most if not all
of it out but if you try to brush it off, it sticks.

Sally H