Lenora, her Pokemon knowing Take Down was a huge pain in the ass. Elesa's two Emolga's Volt Switch were also pretty annoying, though I must say very effective. Thirdly, Burgh's bugs were quite a pain for me as I had pretty much nothing that was effective against Bug types.
The rest was pretty much smooth sailing. I did have some problems with Clay's Excadrill, though I suppose I was just not ready. I must say, never had I so many rough patches through winning my badges as I had in this generation. Was definitely fun though.