Colo gets my vote and yes, I have played both and love both. Wes is BY FAR the best protag out of these two, and IMO the best protag in all of Pokemon hands down. In XD you're just a kid, while in Colo you have this mature, slender figure. And let's not forget- TWO starters! Having both Umbreon and Espeon (one of my favorites by the way) is useful because this game is one of the hardest vanilla games.
The plot is fantastic, you're not mindlessly obtaining badges anymore just because you want to be the champ. Instead, you're thwarting evil bosses and saving the world. I'm sure everyone has tried at least once to snag an opponent's Pokemon before and now that you can is very interesting.
Colo also has a better selection of Shadows. I mean the list goes on for cool Shadows: Houndour, Teddiursa, Bayleef, Croconaw, Quilava, Entei, Raikou, Suicune, Metagross, Tyranitar and a bonus Ho-Oh at the end.
XD has a similar story, but it just doesn't beat Colosseum in my opinion.