Hm, this is a weird situation. I know you must be feeling awful. I know that I wouldn't mind if I found playboys (or the like) but finding an actual pair of underwear is completely disconcerting! Yikes! I guess what you are going to have to decide is whether or not you trust him enough to believe that they are some old (but admittedly strange) memento. If so, maybe you are going to have to decide whether that is something you can accept: him keeping an old pair of someone's panties! If you cannot accept it, then you two are going to have to come to an agreement about it or something that helps you feel comfortable.

How did he act when you asked him about it?

Tell me to shut up if I am asking too much!