Thee's no point in destroying them except for aesthetic reasons. The fungus you see is simply the 'fruit'. The 'plant' is underground, so taking off the 'fruit' will have no impact on that. It won't even make a significant difference to the number of fungus spores in the air.