We used to suck sloe berries when we was kids to try and remove the stain of blackberry juice from our tongues so mother would not know why we were late home from school. I can tell you they are very bitter and leave a dryness in the mouth, whereas the wild damsons we used to gather were sweet once they were fully ripe.
Sloes are about 5 - 10 mm across, while damsons are 15 - 20 mm, but, having said that, if you have a tree that has not been obtained from a reputable source it could be a natural hybrid of anything from that genus.
Provided you enjoy consuming the produce, does it truly matter that much what name someone has decided to hang on it ?.
Personally, I don't eat them, just make flavoured drink instead,
although I use Polish Spirit rather than Gin.