When you add dirt you cannot avoid adding pathogens (unless, of
course, you sterilize the dirt, and I know you don't) and, like
casual sexual partners and STD, while chances are fairly good
that the pathogen you add will NOT hurt your tree, there is the
chance that it will.
As for compost, GOOD compost is self-sterilized. It gets HOT in
a good, well-turned compost pile. Still, compost is pretty fine-
grained soil and since it is the product of decay so by
definition if it has decay bacteria included in it, your compost
will only get finer as time goes on. That clogs up your soil.
It is often forgotten, but roots need more than water and
nutrients. They need air. Clogged soil interferes with the
provision of air to the roots.
Over time, the pine bark we use decomposes bit and actually
becomes a kind of soil in our pot. About then, though, we
repot.