I wouldn't say BW were that easy, because they had their fair share of tricky foes (some of the trainers early in the game, Lenora, Clay to an extent, the final battles with N and Ghetsis, and I also don't think the Elite Four were that easy either). BW2 were even easier, at least on Normal Mode where Gym Leaders still only have three Pokémon, among other things (though in that one I felt it was easier to get caught off-guard and what not).
As for XY, they were pretty easy with the Exp. Share on. But without it, I find that it makes things considerably harder when trying to keep up. In that case, the trainers can catch you off-guard rather easily, because the massive Kalos Pokédex makes it hard to predict what trainers will have. Some of them have unusually bulky stuff early on. In my current playthrough of Y, some of the trainers have been just near my levels, and on occasion slightly higher, so some of them take longer than they did in my first playthrough of X. (I am doing a Monotype currently though, that might be the problem but not much of it.) Interestingly, the trainers are giving me a harder time than the Gym Leaders.