Most people understand English in addition to whatever foreign language they may speak, at least where I live. So if someone starts speaking a language I don't know to me, saying "I'm sorry, I don't understand" usually gets the point across. Otherwise, a quizzical expression works, too.
There have been a few occasions when people speak Mandarin to me and I'm a little embarrassed/don't want to admit that I don't really understand it (I'm Cantonese and sometimes people just assume I speak Mandarin too), so I'll just smile and nod when appropriate, which usually gets me through the short, random conversations strangers strike up sometimes. If I don't feel like it, again, the "I'm sorry, I don't understand" line works too.
If it's on the internet, if I really care about the topic or something, I'll put the text through an online translator. I know they're not accurate, but you can generally piece together the keywords and sort of understand what they're saying.