I walked into the wrong backyard before (trying to visit a friend) and petted the dogs and walked in like I belonged only to find out that it was the wrong house because the people came out and said, "What are you doing here?"

It was awfully embarrassing, but the feelings will pass with time. Now I think it's kinda funny.

I doubt the guy will call anyone. He probably just thinks you were under some stress and made a mistake. Hell, he was probably happy to have such a pretty visitor. And I don't think you can be kicked out of an apartment complex unless you do serious damage or don't pay your rent. Otherwise you can cite the American Disabilities Act and the l.l. will back off.

It'll be okay.