Sad gardenia

I have a small gardenia, "Kleim's Hardy", that I bought last autumn. It's been potted up in ericaceous soil, lived inside for the coldest weeks of winter and otherwise on the patio, where it is looking increasingly sorry for itself. No new growth or even a hint of flowers, and the existing leaves are yellowing or dying completely. Any ideas as to what might be causing it to be so unhappy, and what I might do to help it?

 

Kleim's hardy never worked for me either, survived a couple of winters but was on a downward spiral all the time.