Hi Roughquest
If you simply like your rings, then you should wear them. I am sure you have clothes and books, etc, which were gifts to you from others, whether they be from men, women or ex partners. What are you to do, chuck them away? I think not. Regarding the photos, just put them away, in a box somewhere. I made the mistake of years ago, chucking away all 'my past photos'. I am with a different guy now who is not insecure, and I have no photos of me when I was on holiday, on honeymoon even - all gone, because I gave in. I didn't want them for the other person in them, I want them for me, for the places I went to.
Perhaps you moved in with him too quickly? I think the worst thing you can do is give in, as then it will only get worst. Be firm, tell him what you told us, and then end it with "I love you". Only you know if this relationship is going to work, and if it worth working at. If you are not careful you will end up lying to him, about silly things, just to keep the peace, when you are totally innocent and that will end in disaster. Remember, you do not want to be walking on egg shells in your own home.