I spend quite a bit of time by myself also. By choice. It's just my preference
What you described sounds unsettling. I don't know if it's a mental health issue or just your nature. If it's the former then it's something you can work on changing. If it's just in your nature you can also work on changing it. Or accept that it's part of who you are.
Did the scenario/fantasy that caused you to become upset differ from the real conversation? That seems important.