I don't think that looking down in the direction of his lap likely has anything to do with sexual arousal.

I don't understand why he thought it would be helpful to tell you that he is attracted to you. Saying he would have been interested in getting to know you better in other circumstances may or may not have been helpful, but I don't see the point in talking about attraction. Did he actually say he was attracted or is that what you were thinking based on the get to know you better comment? Just wondering. I would be more concerned if he out and out said he is attracted to you - the other comment does not necessarily imply romantic attraction (though obviously it can).