It's a skimmia. They come in male and female, and yours is female because it has berries.

I'm afraid it's hard to say why the leaves are going pale and falling off as just about any reason that makes a plant unhappy would cause this. Common unhappinesses include too wet, too dry, not enough light and plant disease. I think skimmias are also prone to a bit of this if grown in areas of alkaline soil. I think mine has had a few bits go yellow die off from time to time, but in the long run it thrives and gets bigger, despite being planted in a rather challenging very dry and somewhat shady situation, and me never doing anything except cut out the occasional dead bit.