Well, it wasn't really a "secret club" prior to HeartCatch, but it wasn't quite so biased in favour of one show over all the others. People were generally nicer and more willing to talk about things. I suppose I've just had a few bad experiences with fans of HeartCatch who hail it as Precure's equivalent to Madoka (yeah, you can imagine how I view that comparison) and measure literally every season before and after by their newfound standards...forgetting that this is a show aimed at little girls that is designed mostly to sell merchandise, so why are they taking it so seriously. I'm used to being in the minority with my opinions, so I don't much care if I like the least popular show, but...yeah, maybe I've just had bad experiences with the fandom at large. Fandoms just ruin literally everything, and with something as enjoyable (to me at least) as Precure...eh. It's all the more depressing for it. It stops being funny after a while.
I didn't HATE HeartCatch - the only anime I hate are Accel World, Chaos;Head, and Madoka - I just didn't really care for it. I'm a little bitter because the only interesting one of the four (Sunshine) got absolutely no development, I guess? The art style put me off a bit as well; I realise that was a draw for a lot of people though. It was also...pretty grim, for a kid's show. Fresh was too, but only at one or two points. HeartCatch had Moonlight moping and being generally useless practically every episode. I have a lot of issues with Moonlight. The show would have been a lot better without her in it.
I watch Precure for a break from the general misery in anime, and because it reminds me of the light-hearted fluff shows I used to watch as a kid. I get more enjoyment out of the cheesy stuff than I do the "serious" elements...oh, and the transformation sequences are always fun to watch. I guess that's why I didn't like HeartCatch on an individual note? Just not my thing. Which makes me an evil heretic that needs to be burned at the stake.
But whatever. I would always encourage people to watch HeartCatch last, or not at all. Get an unbiased and fair view of the other seasons first before watching the fan favourite. Even if it turns out to be your favourite in the end, you'll have had a good time watching the others first, and that's something a lot of Precure fans don't do. They watch HeartCatch and that's it; the rest are comparative garbage and that's the end. God, it's annoying...
I've still not watched Yes Precure 5 actually...should do that somewhen.