My ex is still on the mortgage.... For now

Ive had a feeling over the last few months he's been coming in when I've not been here. I've asked him and he's denied it.

I'm going away soon and my neighbor need a key to feed my cats. I asked ex for his and he said he's lost it in his mums house that's why he's never been coming in when I asked.

Should I just change the locks anyway?