I feel like anything with a kind of ELO or rating system brings out some kind of gambler in us. If we lose, then we end up at a spot that's lower than where we would be at had we not even played to begin with. It's kind of that mentality that "motivates" us to keep battling (or doing whatever) to try and get our points back.
With the Pok?mon ladder, as well as some other games (League of Legends for example, something that I have had experience with), it's not entirely up to us to win, as there is a large element of luck involved. You may get bad teammates in LoL. Random Battles are also by definition uncertain. In OU I'd say it's not as luck-reliant, but you may get matched against someone who has a team that wins easily against you. And don't forget, Pok?mon is a competitive game, so the good players always win hax exists. The point is, part of what determines if we win or not is not dependent on how we play, so sometimes we take a gamble. Tilt plays a factor too, we generally tilt more if we have lost more, so we subconsciously rely on these luck factors to pull us through.
It's hard to say what I'll do if I end up losing a lot. It probably depends on the reason why I lost. If I'm using a new team I just try to fix it. If I get haxed a lot I usually quit. If I realized that I screwed up, then I probably load up another game to fix my mistakes. I think what everyone else said is true in the general case - stop playing if you lose. After all, it's not fun to keep losing and it's really just a waste of time. With OU or other tiers at least you'll have a bit of control over the team you bring (I have this one team I ladder with, and it's pretty consistent at pulling out wins), but with Random Battle it's hard to say. Analyze your mentality and see if you're fit to keep battling for the day.