I have had, Cavaliers seem fussy and uninterested. Mine started eating "properly" as soon as I got other dogs, mostly, I think just so someone else couldn't eat it first.
I would take him to the vets, if you haven't already, just to rule out teeth, throat, stomach pain. But he may just not need a lot, especially if he is small.
You seem to have found the answer- he feels safer when one of you is nearby, so maybe feed him in the living room or wherever you are, so he relaxes a bit and add in some fish or chicken or something.
But if his weight, health, energy levels are good, I would let him choose. Mine, I'm sure are absolutely nothing for about 4 years, although there was always food left down for him too... but is an energetic, fit and healthy 9 year old now, so clearly, he was eating enough.