I think he's perfectly within his right to whack off whatever
part of the plant he feels is invading his yard, but if he's going to do
that he can take care of the debris himself.

Legally, he is perfectly correct. The plant is your property, but it is
trespassing. He is within his rights to prune whetever sticks over the
preoperty line, but he may not keep the trimmings as they are your property, as
they came from your plant.

If you talk to him and tell him he can keep whatever he cuts he MAY dispose of
it in future, but he is under no oblgation to do so.

If he keeps the trimmings he commits theft unless he has your permission.