The Olympics are like crack to me; especially the Winter ones. At the most recent games in Vancouver, my housemates and I got seriously addicted to all of the cross events and anything involving a bobsleigh track, despite the fact that none of us had ever gone skiing/snowboarding and had no idea of the rules. With certain sports like that, it's less about rule comprehension and more about just getting into the moment. A few crashes along the way doesn't hurt either!
As for the rest of the time, I can watch a good game of tennis, as each game is normally just a quick burst of energy. It's like watching lots of little games instead of one big long slog, as other stuff like football (soccer) or rugby can often be. Sometimes, I'll also watch Formula 1, less for the actual racing and more to witness the strategy involved.*
And yeah, for the crashes... I'm only human, dammit!