Yes, salt build up can happen very easily if the water you use is high
in dissolved salts. You can salt-poison a plant very easily without a
whiff of fertilizer.

When is it time to repot? If you knock the plant out of the pot and
there are live roots circling the pot and the root ball takes up the
whole pot, time to go up one pot size (ca. 1" all around.) If the
soil is waterlogged and foul and there are dead roots, time to trim up
and repot with good soil in the same size pot. (Wash and sterilize
the pot first.)

M. Reed

Alan Little wrote:

Carved in mystic runes upon the very living rock, the last words of
Monique Reed of sci.bio.botany make plain:

For both of your plants:

Check for salt buildup in the soil. Water should run clear from the
pot, not colored.

Would this apply if I haven't been using any fertilizer?