It sounds like you have a pretty legitimate excuse to feel jealous...I think that even a person who didn't have a past history of jealousy in a relationship would have some kind of feelings in that situation. Its great though that you are trying to not let that jealousy take over.

I would suggest that you talk to your boyfriend and his ex and explain to them how you feel. Maybe suggest that it's fine if they want to hang out together-but you'd rather not join them (you trust them right?). If you don't trust them to hang out alone together-maybe you do have to reevaluate your relationship.