Blue's my favorite rival, I like the personality he has in R/B/Y as well as him having all the badges (before you). Silver apparantly wasn't that interested in the gym battles and therefore it felt quite odd to always be one step ahead. And of course the fact that Blue became a gym leader in the second Generation gives him a significant role in both games.
Also, Silver is way too weak level-wise in (almost) every fight. I mean, it was rare when my levels were actually lower than his levels, and I played the game a lot. And no, I didn't train my pokemon a lot vs. wild pokemon, I just beat trainers. The first battle vs Silver is no big challenge as your level usually is higher. The second fight in Azalea Town is in my opinion the hardest fight vs Silver. But the only one that's actually difficult to win..
Also, I think Blue is more like a Rival than Silver, because he has the same goal like you have - become the best trainer by winning 8 badges and the E4. Silver however, even though he states that it is his dream to become the best trainer, I also think that another goal of himself is to find his way. That's why Silver's story has more depth, but my favorite rival is Blue. While the levels of Blue's pokemon have raised significantly, which makes him a really tough enemy, the levels of Silver's pokemon in the last two fights are just pathetic. Blue, the last gym leader, has pokemon above level 50, while your rival's pokemon are not higher than 50, and I don't like it. Also the battle at Mt. Moon must be the best example. All Gym Leaders you beat till that point (except for Janine) have higher levels than your rival.. that's what I think is pretty dumb. My last point is that I like how Blue's team varies depending on the starter you/he take(s), which makes his team more interesting. I don't like that the team of Silver always stays the same.