Gen I: Probably Misty, especially on my first couple of playthroughs where I'd only use my starter, Charmander. Brock wasn't too difficult, but Misty and her Starmie always gave me trouble.
Gen II: Whitney, 'nuff said.
Gen III: Norman and his Slaking gave me the most trouble, IIRC. I can remember strugling against Wallace/Juan a lot too, although not so much.
Gen IIII: I remember Gardenia giving me a bit of trouble (I picked Piplup, can you see a pattern now? XP) I can't remember much about D/P/Pl, to be honest. :s
Gen IIII: Ah, this is easy. Clay. Seriously, every other Gym leader was a breeze, and th ONLY one I lost against was Clay. I think I lost twice, actually.