Personally I would not say it that soon, and I wouldn't like it said to me that soon. I would feel like they were going way too fast.

However, that's just a generality. All relationships are different. If it feels right, say it. If you think he might be put off by you saying it, then you might want to hold back. You could maybe judge his reaction to things by bringing up general topics like what do you think of love at first sight, or something. Some couples do meet, fall in love, and have a successful life together after just a week. Some take much, much longer. It's all up to how it feels.