WovenGalaxy, you say the relationship is very new and it may seem that he's trying to rush things by inviting you for the weekend. I can understand your apprehension about meeting his daughter too. Maybe ask him to visit you again, so that you can have a private discussion about what concerns you.

The comments he's passed may be due to nervousness, or he has that kind of humour that only he thinks is funny. You'll probably only find out by persevering with the relationship to see whether that aspect changes or stays the same.

His daughter probably meant well with her comments, as it's not unusual for children (of any age) to be hostile to their parents' attempts at romance/finding a new partner. It happens when they're much older than twenties too! Perhaps she's also direct in her manner, without considering it could make you feel uncomfortable so early in the relationship.

At the end of the day, it's your gut feeling about him that counts. If you continue to have these nagging doubts then it's probably better to move on.