It really depended on what Pokémon I was using but...
Roxanne was never really hard.
Brawly, like Bobandbill said, had an evil Makuhita and always used a Super Potion when I was about to win.
Wattson was only a problem when Magneton resisted all of my Pokémon's moves.
Flannery was only hard when her Torkoal got Attract and paralysis on you and then hit you with a sun boosted Overheat.
Norman was really annoying, unless you had a Pokémon with Protect because Slaking hit hard.
Winona's Altaria was the only thing to worry about with her, Earthquake was really annoying.
Tate & Lisa were really hard for me recently in a Nuzlocke, I was lucky to have a Sharpedo to sacrifice.
Wallace and his Pokémon healing themselves was so annoying.
And the E4 are generally little problem, seeing that I always end up overlevelling my Pokémon by then. But none of the trainers are really that difficult, these are just the annoying aspects.