it is a tall grass like plant with spikey stalks and little
brown balls that grow part way up the stem....

Although I have a book giving the wild flower key by Dr Francis Rose
(revised and expanded 2nd edition), the info you give is insufficient to
identify the plant accurately but I believe it to of the allium family.

How to get rid of it? Keep cutting it down to ground level and/or keep
digging!

Lionel