Nobody knows what circumstances lead these people to the kind of lives they have now, but if unlike most of us, they were just unfortunate, then you definitely did the right thing.

I live in NYC, which doesn't lack homeless people either, and like you, I find myself saying no too many times. So sometimes, I just act on instinct and give away that Valducci's Original pizza slice or Boston Kreme Donut I have been craving all day, and it actually feels better than satisfying my hunger.