Thanks for all the advice.

My pond is a wildlife pond so I don't need to worry about fish. As it is new it doesn't have a lot of dead material in the bottom and in place you can still see the liner. Would it be best to let the plant rot down to provide bottom cover or take them out?

I must admit the lillies are still in the 'stupid' pots as I haven't repotted them. The pots are actually quite big so I thought I would do it next year when I would probably need to split them? Is there any advantage to doing it now?