If he really really really likes you a lot, and respects you, then he will break up with the other girl to be with you.

Right now he is disrespecting both of you with this.

The right thing would be to not see or speak to him till he breaks up with the other one - right now he's "having his cake and eating it too".

Don't worry about him feeling guilty...worry about doing the right thing.