I'm not gonna count the XY group in this because they're meant to be your "friends" and not rivals. But if they were considered as rivals I'd probably name them as my least favorite because I never enjoyed when they showed up. Ever. XY's actual rival, Calem, was disappointing in the character development department but still bearable so no one from XY takes my pick.
In a similar vein, (ORAS) Brendan isn't quite your rival but if he was ohhhhh boy he'd be my #1 pick I absolutely cannot stand him.
I know this is a really unpopular opinion but I guess I'd have to go with your gen 1 rival (Blue). Silver is awesome and has great character development, Wally is great (especially in ORAS), I like Barry because I think he and his personality are pretty cute, and all the rivals in gen 5 are spectacular. Blue is just kinda mean and doesn't have any sort of growth throughout the game--I mean, it works, because that makes you want to defeat him even more once you make it to the Pokémon League, but in the end he feels kinda bland when he has no development and Silver filled the whole "nasty rival" role better and had character development on top of that.