That's just the point. He's miserable when people _force_ him into social situations that serve no purpose other than socializing. He's not miserable when he's alone.
I'm not sure why this is so hard to grasp for a lot of people.
The only thing that would force you to go to a party is not wanting to disappoint/irritate people. If the author doesn't care about having anything to do with other people so much then that shouldn't bother him.
I'm not sure why this is so hard to grasp for a lot of people.