I agree with riotgrrl. Also,

(1) you called him a "new guy I have been talking to/seeing" - normally, as a matter of rule, people do not "love" every new guy they have been "talking to" - love at first sight is an exception. So, that.

(2) "ADHD, bipolar, borderline personality, and schizoaffective" - people cannot have bipolar and schizoaffective, but can have either or. So there is some confusion in his mind about his diagnosis, which he transferred to you. Whatever his exact diagnosis is, those are serious disorders (ADHD perhaps less serious than the rest in terms of relating to people) and you should step back, read up on them, and assess whether you are prepared for such a handful. If you are not, it is perfectly OK to say so in a polite way and stick to your guns.

(3) he is probably on a "locked unit" (the more severe type of psychiatric inpatient) since he is only allowed one call a day. That speaks to the severity of the concerns, which calls for doing even more of (2) above. Also, if the hospital is that restrictive, you probably would not be able to visit, but you should inquire. Do you know which hospital and can you call the unit to inquire about general rules for visitors?