pick off and squash or spray with bug killer. these sound like red lily beetle to me, if you can shake the plant without breaking it, they fall off on to the ground to be squished.
I had them on all my lilys last year, but luckily I have`nt seen a single one yet......yet. You must creep up on them without casting a shadow or they drop off and land feet up so that you can`t see them.