Certainly not aware of the larger things happening, and you are right, he won't remember before, but the separation does affect him. But there is no reason he can't have a perfectly normal happy childhood.