Whats wrong with these Foxgloves?

Pictures 1,2 and 5 are plants we put in last spring, they flowered well last year but the seem to not be doing very well this year and lots of little holes all over even the new leaves.


Picture 3 is a lupin from last year that has some leave problems also.

Picture 4 is 2 new lupins we planted 2 weeks ago, they seem to have leave problems already.

I don't see any real pest problem, it looks like somethings been eating the leaves but never seem to find anything. I do see ants in the flower bed sometimes though.

Not sure if its a pest problem or a disease problem, can anyone suggest what to do?