I'm not gonna lie, I am obsessed with Pokemon. I have plushes, I play the games, I watch the anime, I have the cards... And I play and discuss all of these online commonly. Most of my free time is consumed by something Pokemon, whether it is the /vp/ board on 4chan, or the subreddit. All of my wallpapers on my computers are Pokemon, I have Pokemon clothing, and I have figures that sit by my desk at work as I hack away at things.
However, I don't always talk about it with people in real life, I don't let it interfere with people I work with or hang out with, and I make sure that I'm never too overbearing to the people I'm around. If you pass me on the street, you probably couldn't even tell I was interested unless I happened to be wearing a shirt that day. I am sociable, and have many friends who are not into Pokemon, many who despise it even...
The way I see it is that as long as your interests don't affect your interaction with others, you can like anything as much as you want without it being a problem. Many people here will say "screw what others think", which is nice and all, but people are very social. Even introverts want to have meaningful interactions with people every once in a while. But if something you like lessens your ability to effectively be with people, then that is a place where you should try to limit yourself. But, as long as you can still remain competent within other aspects of life, how you spend your free time is entirely up to you, and if you happen to enjoy something enough that it can use most of your free time, I see no problem with that.