If you do decide to remove this tree, it's quite possible that the tree
may re-grow from the stump.

I had my first veggie garden close to a HUGE Willow Tree. Even if a branch
broke off from it and fell in the garden, it would start growing.
Best to get rid of the entire tree, stump and all.
Sue