It'd sorta break tradition, though this is BW, and traditions could easily be broken. Still, to have a tournament where the opponents are randomly generated to stay fresh… you'd still need to have memorable people. I mean, the Elite Four itself are pretty memorable since they have personalities and such, and trainers… mostly don't.
I dunno, I think if there was no Elite Four and just a tournament challenge in the end, it'd mean several painful battles to get to the top than just five. Rather than replace them, what if there was this whole "once every ___ weeks/months/years" associated to a sidequest sort of tournament? That'd make it feel interesting. On top of that, it'd add more than the Elite Four to the "ultimate goal" of the games.