I shouldn't think you are alone to be honest, and as for "wrong" I don't see why there should be a problem (only if it takes over your life, ruins multiple relationships, etc, which doesn't seem to be the thing here). The only "wrong" fetishes imo are ones that involve lack of consent and so on where there is actually a moral issue or where the fetish itself is a real problem.

Why was your girlfriend jealous? If she felt that you were paying more attention to the stuffed animals than to her, then I might see her point (unless she's getting it out of proportion). It does occur to me that with an understanding partner and a bit of communication, you probably could integrate stuffed toys into the bedroom in some ways that might be enjoyable for two people. Maybe not all the time though if that's not her thing. Perhaps you could think of a few sensual things to do to her with them and change her mind a little?