Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I would be very surprised if a simple bag-of-words approach works in this case. Intuitively, it's not the presence of certain groups of words that's important, it's something much more subtle and structural. Something that might be promising (and I'm being very handwavy here) is to discover 'template' sentence structures, as well as the particular words that populate those templates that result in TWSS.


I don't believe templates would work very well. The variations of sentences are too great such that you will result in very low recall.

An alternative solution is to attack the problem backward, training on terms (words or phrase) from sex-related conversations (such as adult chatroom transcripts). Then, from general corpus (Twitter or generic chats) identify terms that highly cooccur with those sex-terms. I would still use a Bayesian classifier, with strong prior against labelling something as a TWSS.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: