I have a tea olive shrub (osmanthus fragrens) and need to know how to prune
it.

First, the shrub is far out of its natural habitat of the lower southeastern
United States. I grow it in a large pot which I bring in during the coldest
months.

It's presently about 4 feet all. I'm thinking of cutting it back to about 2
feet, with the hope that it will come back thicker.

Is this a good idea?

When should I prune?

There's a photo at the link below:

http://mysite.verizon.net/vze4xpgq/teaolive.htm