Unless said actions are explicit, I wouldn't jump to conclusions, especially based on one action. For instance, if my friend made a joke about me not getting an A in certain subject bringing disgrace to my family or whatever, I wouldn't automatically assume he's a racist.
Edit: I think I need...