You have to be careful with what is "racist" and what is not. Some people for example think I'm racist for using the term "black" as opposed to "African American" (which I don't use because they may not be African in decent or American in current nationality), for example. Some people at work thought it was racist when I felt bad for black families who came in to take photos on the Christmas background a few months ago, only to show off so dark that it's difficult to make out their faces. I don't get some people.
If there is an obvious hating intent, I don't have anything to do with those kinds of people. Period. But I think that what counts as "obvious hating intent" or "violation of political correctness" are two totally different things though.
A related example, it's like my best friend is a blind guy. I make jokes about it all the time - ranging from him shooting lasers out of his twitching eyes, killing everyone, to dumb drivers running him over. People stare at me and act like I'm an ******* but he's laughing with me, and I'm there for him when he needs help from me for whatever reason, and honestly isn't that the kind of **** friends do? It's our way of playing around with each other and it makes us both feel natural with each other. Political correctness is lame.
It's the same thing really with racism. I will bet you that much of the time when people are being "racist" it's more like they're violating political correctness and don't genuinely hate a certain race or whatever.