-Double Team. Switching out your Pokemon won't do you any good in raising the accuracy of your hits (like it would with moves such as Sand-Attack and Smokescreen, that affect your Pokemon's accuracy rather than the foe's evasiveness). After five or six of these get racked up, you'll be wishing you had a perfect accuracy move at your disposal…
-Confuse Ray. I. Hate. Confusion.
-Attract. My Weepinbell was once attracted by a Miltank, and failed to land a good 80% of its attacks. 50/50 chance... right...
-Moves that cause poisoning. I don't care if Poison Sting has a 20/30% success rate of poisoning... used against one of my Pokemon, it may as well have a 90% success rate. XD
Protect doesn't really infuriate me as much as get on my nerves for wasting turns. Listen, Pelipper, you know you're going to get owned with Thunderbolt next turn – stop delaying it. Of course, there are some situations where it can really annoy me – mixed with Double Team/Sand Attack and a poisoning move, for example…
I also can't stand Swagger. I love it when Swagger backfires, though. I once had an opponent use Swagger on one of my Pokemon, and it just happened to be holding the proper berry to heal confusion. The opponent was regretting that move, that's for sure.