Can you have a casual conversation about his plan to move? The next time he mentions moving to the Caribbean, be curious and ask how long that's been a dream. A serious conversation may spook him, but a conversation on a getting to know you level may help you get a bit more info without revealing your level of interest.

There's a difference between wanting to do something and actually doing things to make it happen. It sounds like youneed to ask a few questions to find out if he's dreaming or actually planning and doing. You don't need to reveal any of your feelings or fears to show an interest in something he's mentioned a few times.