People in those positions are not always people who lived life carelessly. Maybe I'm saying that because I live in New York City, a city where you could be living in a house one week, and on the street the next with relative ease. But I find it very ignorant to assume that the people begging for money are trash. You don't know their story, so don't be a know-it-all moron and assume you do.
That being said, I only do when it's very obvious that the person is indeed in need of help, and even that's risky. In NYC, it's just as likely the decently dressed fella is indeed homeless, whereas the guy who looked like he ♥♥♥♥ himself just wants some easy money for the day. I was fooled once, and that left me a little more cautious. That's panhandlers, though. There are also performers who do some crazy shenanigans and ask for money. I usually don't give them money, but I have nothing against their 'business'. They are, afterall, working for their money.