Pardon me if I'm telling you what you already know, but . . . .

Most cacti need lots of sunlight and heat, very little water, and not
too much cold. Many of them can go for several months without being
watered. Really. They do better with neglect than with
overattention.

You said you brought the plants inside. I take that to mean they're
in pots. Either way, whether in pots or in the ground, cacti
absolutely need to have good drainage, and not too much organic
material in the soil. They don't need "sand", necessarily, but if you
put the cacti in peat moss or clay, or if you water them too much,
they will rot.

Then there's the usual varmints (bugs, etc) that will nibble on
anything you have, anyway.

And about growth: some cacti do grow slowly. My mom had a saguaro
that grew to a height of 2 or 3 inches in ten years.

If you can tell us where you have put your cacti, that might help to
clarify the problem. Do you live in Yuma (great cactus climate), or
in Seattle (a great place to kill cacti)? Do your cacti have sun,
shade, rain, fungus, etc?

Ted Shoemaker
certified grey thumb