Hi Seeker,
I think you are doing the right thing.

People view sex in different ways. A casual "fling" might make one person happy, while it can be downright destructive for someone else.
If it is a relationship you are seeking, I think it is better to connect with someone who wants the same things that you do. Of course it is up to you what you should do, but you must follow your own instincts and be true to yourself.
Love,
Zen