Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
In Screening for Suicide Risk, Facebook Takes on Tricky Public Health Role (nytimes.com)
75 points by mindgam3 on Jan 1, 2019 | hide | past | favorite | 100 comments


I've been hospitalized twice on a suicide watch. I've also had supposedly "well meaning" strangers call the cops on me when I was homeless to "check on me."

The cops advised me I shouldn't be walking in the bike lane that everyone walked in because there was no sidewalk on that side of the road. Having advised me of this, had they stopped me again, they would have ticketed me. I then had to routinely expend extra energy to go out of my way going "home" every single day.

No, the cops absolutely weren't going to help me solve any of my actual problems. No, they absolutely didn't care about me, no.

Calling the cops on a stranger because you supposedly care about their welfare is asshole behavior. I don't care at all about your supposed justification or algorithms etc.

If you care, you check on them yourself or maybe call an ambulance. If you don't know them well enough to do that without it going weird and shitty places, then mind your own damn business.


That's terrible about the policing of walking in the bike lane. It reminds me of the woman in Georgia whose child was hit and killed by a driver (who had been drinking) in a hit-and-run when she crossed a road without a crosswalk. There was no crosswalk nearby. She received a much harsher sentence for crossing the street without a crosswalk than the driver received for the hit-and-run. https://usa.streetsblog.org/2011/07/14/mother-convicted-of-v...

Our institutions are failing us. https://www.ted.com/talks/barry_schwartz_using_our_practical...


I just read the first link. Disgusting. She crossed the street with her children and saw one of them run over in front of her, and she was charged? And did more time the one who ran them over? Wow.


I guessed the usual reason for absurdly punitive judicial decisions in the the American system, and after finding a photo of the mother I was sadly proved right.


In mostly places there's no difference between calling police or an ambulance. Everything is handled out of the same dispatch center, and the dispatcher will send whichever first responders they think are appropriate based on policy.


> In mostly places there's no difference between calling police or an ambulance.

At least in Germany, there historically is/was (even though this is not always the case anymore as far as I am aware). 110 for police and 112 for ambulance service and fire department.

There is a good reason for this: if you and a group of friends did some experiments with, let us call it, substances that did not go well, you want to call an ambulance to get them medical help. But under no circumstances you want that the police to be informed of this accident.


I didn't actually like it when strangers called an ambulance on me when I was homeless either. But the ambulance driver at least didn't threaten to ticket me if he ever saw me again in that spot.

Also, the remark about ambulances wasn't in my original comment. I added it after the fact, perhaps a mistake.


Former EMT. If you called in that you had suicidal ideation and had the means to harm yourself, it would be treated like a hostage situation, where you are both the hostage and the captor--especially if you have a weapon like a knife or gun.

I can see few situations where they would send only an ambulance. The closest is if you said you simply overdosed on pills. However, at this point you need to go to the hospital. The ambulance has no ability to force you if you change your mind. Police would likely be sent along to make sure you didn't change your mind.


Some places they habitually send a cop and an ambulance, even if there's just a medical emergency.


firstnet.gov - Now that you mention it, I was browsing a topic, and one thing led to another to another and led me to firstnet.gov. But, you're right. Where there's an ambulance, there's police.


[flagged]


Assholes usually don't actually care. They just like to imagine they are good people and that their latest decision to fuck with your pathetic life while not actually genuinely caring about you at all is evidence of what wonderful people they are.

I wish assholes would stop using me as evidence of what wonderful people they imagine themselves to be. It only compounds the problem.

I have yet to run into anyone on the internet who actually cares about my actual welfare when I talk about my struggles with being suicidal. No, it's just some societal trope that trying to keep me around is some kind of good behavior on their part, never mind that they will not do fuck-all to help me resolve any of the real world problems that make me suicidal and probably won't stop fucking with me for giggles and out of prejudice either.

Assholes who imagine they care are the absolute worst people I deal with in life. There is zero upside to dealing with their shit. Assholes only produce shit. That's probably why the world calls them "assholes" to begin with.


I think you’re wrong. An asshole who cares at least has a moral compass, and knows the direction it should point. They lack motivation though to go above and beyond their own selfishness to do more than the bare minimum, but there is still hope that one day they will.

An asshole who doesn’t care at all has no hope. They truly don’t give a shit, and will only pretend to care as long as it benefits them. These are the worst.

If a homeless person is dying in the street, you really think it’s better for an asshole to just step over them and continue on without a second thought, rather than say calling 911 and then continuing with their day, just because you feel their level of care is too low?


We aren't actually talking about someone literally stepping over a body on the street. We are talking about Facebook deciding to fuck with people by calling the cops on them.

But let's go with your theory.

I was homeless for several years. I also struggle with being suicidal due to my intractable personal problems that were the root cause of my homelessness.

While homeless, I spent time successfully developing an online income. It was an uphill battle because of the shitty way I was treated by people online who do know how to make money on the internet.

Such people, no, were absolutely not willing to give me useful constructive feedback to help me develop my projects. Some of them only ever spoke to me to be openly contemptuous while pretending that speaking to me at all was their good deed of the day because they are that monstrously classist.

If someone is homeless and dying on the street, a helluva lot has already gone wrong and the lack of positive, constructive social connections combined with a preponderance of negative social connections will be a large part of the problem there.

When I was homeless, people sometimes felt compelled to try to say they didn't want me to die when I was feeling suicidal while they were incapable of not shitting all over me. I didn't need some asshole trying to use me as their imagined good deed of the day.

My experience of such moments was that I was being held hostage and compelled to continue to suffer by people who were going to continue to treat me abusively and not do fuck all to help me, or even fuck all to try to figure out how to shit on me less.


There are plenty of jurisdictions where calling 911 on a homeless person will make their situation worse - resulting in their property being confiscated and/or destroyed, for example.


The one time someone called an ambulance on me, I had lain down in the grass in a public park next to the sidewalk. I was walking somewhere and I have respiratory problems. I was having respiratory distress of a sort that doesn't resolve with sitting, but resolves handily if I just lay down for like ten minutes.

The only evidence that that there was a problem was that I was homeless and laying down. I had no collapsed. I was not thrashing about. There was no blood or foaming at the mouth or whatever.

Most likely, the reason they called an ambulance was pure prejudice and fear of approaching me to say "Hey, are you okay? Do you need help?" If I had not been homeless, they probably wouldn't have called an ambulance at all. They probably would have spoken to me to find out if I was in real distress or merely choosing to catch my breath.

An ambulance was also called once when my son laid down on the side of the road to rest while walking cross country. Locals saw strangers in their neighborhood and freaked out and called an ambulance.

In neither scenario was there any evidence of real distress beyond, well, we were obviously poor. The thing they had in common was we were poor and people were afraid to approach us because, obviously, homeless people are all junkies and crazies and will do something violent if you speak to them, I guess.

I spent most of my life middle class. I lived without a car for a number of years before being homeless. When I laid down in the grass to catch my breath while middle class, someone asked me "Are you okay?" rather than dialing 911 to sick the authorities on me. I told them I was fine and waved them on.

The irony: I was much, much sicker that day, having thrown up for 45 minutes in a public bathroom before continuing on my walk. But I was pursuing alternative remedies for my condition and walking was a big part of that and I wanted to be left the hell alone by the medical establishment has informed bluntly that people like me don't get well. So, no, I didn't want to talk a doctor about things that day. I wanted to be left the hell alone to pursue remedies that were working, remedies I was afraid the authorities would interfere with if I had to explain myself to them.

I also threw up in the bushes one day while homeless. An asshole yelled at me out their car window as they passed "Drink too much!" I don't drink. I do have a serious medical condition and sometimes my lunch doesn't stay down long because of it.

No, most people don't give a flying fuck about homeless individuals and their welfare. In most cases, they are just prejudiced assholes looking to fuck with the homeless. That's it.


I just want to say I'm really shocked that you would continue to argue with this person, who obviously has dramatically more experience with this particular topic than you do. What exactly are you trying to do, prove them wrong?

You also put up a really simple strawman with "homeless person dying on the street". That's a really contrived scenario and not what actually happens, as the person you're replying to has already explained.


This. Isn't. Facebook's. Job.

No, seriously. Facebook should host content and serve ads. That's it.

They don't need to worry about being the arbiter of truthful news.

They don't need to stop "hate".

They don't need to make calls about who needs a psych eval.

They just need to host content. But if they truly want to do this, what happens when someone sues them for not making the call?

I made the new year's resolution to delete my account yesterday. I almost chickened out, but then this article comes up.


I mentioned in a previous HN thread that there likely wasn't a line Facebook could cross that might make me delete my account. I'm not there yet, but I'm willing to admit before that I was wrong. If I posted on Facebook, I would leave the platform here and now.

I've had two friends who went to take advantage of university counseling services for anxiety and depression that they wanted help with, only to be committed to a local hospital involuntarily for saying the wrong thing until their families intervened to have them discharged. The episode would have merely been stressful for them, but they were each restricted from returning to their classes until the next semester. Neither of them were at any level of risk for suicide or self-harm, but the universities they went to naturally were obligated to err on the side of caution.

Such events are relatively rare, but keep many students who are suffering from mental illness from taking the risk of using available resources.

I'd honestly hate to be committed because I was a little too sarcastic in a Facebook comment, and much like public universities I fear Facebook is incentivized to make the least risky call.


> Such events are relatively rare, but keep many students who are suffering from mental illness from taking the risk of using available resources.

Even more: Exactly these kinds of rules lead to a strong incentive to simply commit the suicide because talking about it, will lead to worse consequences.


In the US, state laws strictly regulate who can and cannot be involuntarily committed to psych wards. Staff at the university really can't be any more "strict" or "harsh" than staff at any other councelling center.

In recent decades, psych-laws have become more strict because as a society we have decided that it is better to involuntarily detain 20 people for a few days rather than let one person end their life. That seems like a pretty fair tradeoff to me, although one that reasonable people could easily disagree on. If you disagree, you should be lobbying your state psychological association, not complaining about random therapists at colleges.


States often allow psychological holds and extensions.

There’s little to no evidence required to keep someone for days or weeks under “observation”. Some states setup entirely separate court systems without due process, since commitment is a “civil matter”.

Some amount of blame falls with states allowing this type of behavior.

The burden is on psychologists to demonstrate the practices work and hospitals are properly equipped.


I still might chicken out. There's a lot of people that are likely going to fall off my radar. The thought makes me sad.


Don't get me wrong. Leaving isn't worth it yet. Still quite a ways off.


Least risky to who, exactly?


Facebook does not just host content.

I would agree with you that Facebook should not be an arbiter of truthful news or stop hate.

But they are indeed an arbiter. They continue to push content to users to keep them consuming their product. They promote content that makes people engaged and that usually means content that makes people angry.

If they stop inserting posts from sources users don't follow and not rank stories from friends, I would agree with you.


They also host advertising. I wonder how much advertisers would be willing to pay for suicide risk flags.


I hear you... But what if they have a reasonably accurate model that tested well (say 75%).

Once day, their researchers decide to test how accurate the model is by setting real time flags, but take no action. And let's say, it turns out, of 100 people, 75 people did end up taking their life.

What would you propose they do?


Being able to do something is completely different from it being an advisable thing to do, and this is a can of worms that would be better left untouched.

For example, if Facebook can identify suicide risks in Ohio, then they will be pressured to identify gays in Saudi Arabia and Muslims in Xinjiang. Or weed smokers, underage drinkers, and a bunch of everyone-does-it-but-nobody-cares crimes, where the punishments are technically quite harsh.

The only effective standard that differentiates suicide from the others is "what Facebook's executive thinks is morally acceptable". It is certain that Facebooks executive are not the right people to arbitrate moral issues, not matter how black and white.

There best layer of protection we have here is Facebook taking an official position of "we don't judge, we just sell ads". It isn't perfect and it isn't necessarily logically consistent, but if Facebook sees itself as an arbiter then sooner or later it may well disagree with something we do and take action to stop us doing it.


Say there's a car manufacturer that makes a car. It's a reasonably safe car, but sometimes people speed in it.

Without really telling anyone, they install a webcam and point it at the driver to see whether they might speed on a certain day, based on facial expression and skin color. Why those? Just a weird hunch! They find that the person actually did speed. (Turns out, they didn't really tell anyone that all your driving data was uploaded to their servers, too!)

The problem is two fold. First, they shouldn't have been collecting this data on their users in the first place. The manufactures job is to build a car, not ensure that the car is being used responsibly. This is simply an impossible task to charge anyone with, and the implications are frightening.

Second, I'm not a believer in using algorithms to predict behavior. Too much black box "mangle the numbers until it works". You could make an argument that there's a correlation between skin color and speeding. You might even be able to make pretty charts, but I think this is simply bad science.

You make it sound like this model would actually continue to work at 75% perpetually. I think that it's far more likely it worked this one test, and performance would trend sharply downwards. However it makes this decision, I think it's about as absurd and wrong as the crazy example above.

On the other side of the coin, people are putting too much trust in magic algorithms they don't properly understand, and are far too complacent about egregious invasion of privacy.


There are any number of measures we could take that would prevent deaths. We could lock people in a cage and monitor them 24/7. That would keep them alive. Should we do it?

# of deaths reduced is not the sole factor we should use to determine whether a particular course of action is good


Agreed with this. In fact, I feel a lot of the issues society has with privacy and freedom of speech right now come from how it seems to be willing to trade virtually all freedom for the illusion of 'safety'.

A free society is also often a risky society, but the rewards of freedom outweight said risks.


I would have them either enter the mental health and public safety industries or I would have them throw the algorithm away.


Nothing, because Facebook should not be in the business of making these decisions.

For sone folks, like me and the GO, there are no “buts” about it.


It isn't facebook's job. They can choose not to do these things.

But I'd rather that they did do these things. They have an opportunity to make some people's lives better by doing things like this and I think it would be nice for them to do that.


With great power comes great duty.


The officer who took the call quickly located the woman, but she denied having suicidal thoughts, the police report said. Even so, the officer believed she might harm herself and told the woman that she must go to a hospital — either voluntarily or in police custody.

It’s an imperfect predictor and Facebook will make a mistake which will eventually be followed by a police shooting. After all, how many stories have been published of police called for mental health issues that resulted in the shooting of the person the caller was worried about.

I hope the programmers are ready to explain how it worked because I get the feeling the family’s attorney is going to have an easy time with black boxes, machine learning, and no true tracing of why.


Regardless of the reasons for the call, can you really pursue damages from someone who made a good faith call to emergency services?

The only legal issue I could see is violation of privacy, but I bet their EULA covers them there.

Now, if they screw up bad enough, I could see it precipitating some new legislation that they might not like. Until then, I think they're in the clear.

It's another one of those situations where everyone is incentivized to "do something" whether it's actually helpful or not. "Student dies after symptoms of suicidal depression went ignored" is a splashy headline. "Student unable to graduate after involuntary psychiatric detention" is not.


As litigious as our society is. And having been involved in wrongful death suits in a previous line of work(firefighter/emt). When somebody is looking for money the swath of people who get sued is as long and as wide as it takes to find deep enough pockets to get some money.

Facebook isn’t exactly the most trusted and liked company right now. So they are exactly the type of entity a lawyer would love to get in a court room.


Google "a bias for action" and you'll see it defended as something good.


Regardless of the reasons for the call, can you really pursue damages from someone who made a good faith call to emergency services?

Yes, you can try to sue for anything. Also, yes, because the lawyer will try to show it was not "good faith" but a failure of software that caused the tragedy. I'm sure the discovery stage will yield enough to make it painful for Facebook. The current feelings about Facebook are going to come up.

A EULA will not protect you from the families of dead people.


> Yes, you can try to sue for anything.

Point taken.

> the lawyer will try to show it was not "good faith" but a failure of software

AFAIK, the phone calls are being placed by Facebook employees, not software. Presumably, these employees have to look at the actual post content to even be able to make a coherent phone call. You would have to argue either that the employee was incompetent, or that the employee was coerced into making an inappropriate call.


What’s the chance that the employees are licensed social workers or psychologists? Also, the software used in the initial data mining is going to come up. I expect discovery is going to be really painful.


This sounds like swatting.


Didn't think of it that way, but yes it could escalate into a unintentional swatting-style situation. I'm betting some newspaper will call it just that when the first tragedy happens.


Maybe the police should stop shooting people...


Google "suicide by cop", it's a real thing.


And notice how it's not really a thing in Europe, where the cops choose to not constantly shoot people.


People are unarmed in Europe, so shooting at them is almost never necessary. Meanwhile, the cops in America are constantly one pull of the trigger from their early grave, which I imagine is incredibly stressful and also, in fear of their lives, makes them more likely to assess ambigous situations as life-threatening. Compared to most of Europe, I imagine being an US cop is closer to serving in a war zone.


> People are unarmed in Europe, so shooting at them is almost never necessary.

Has it occurred to you that cause and effect are non-obvious?

AIUI part of the reason most UK police does not carry a gun is precisely a policy of de-escalation — if the police doesn’t carry a gun, criminals don’t feel the need to match their firepower. Inversely, the fact that American law inforcement treats their beat like a warzone probably contributes to an escalation of violence on the other side too.

Same concept different application: escaping prison isn’t a crime in Germany. You will still be chased by law enforcement to serve the rest of your sentence but the act of running away isn’t a crime unto itself. This means an escapee is likely to avoid committing further crimes on their way out to avoid worsening their sentence, while an American escapee, facing a large penalty for escape itself, is likelier to take an in for a penny, in for a pound approach.


> if the police doesn’t carry a gun, criminals don’t feel the need to match their firepower.

I'm not convinced. Committing crimes, esp. robberies, is much easier with guns. Not to mention that a lot of criminals are involved in intra- an intergang relations and in such environment, a drug dealer without a gun (for example) wouldn't last very long.


I think in America, it is too late. The arms race between police and criminals has gone on for too long. If police stop carrying guns, criminals aren't going to magically stop carrying the guns they already have.


Regular reminder that "Europe" is a big place, including such countries as Switzerland where guns at home are a normal part of military service. Several other countries have strong hunting cultures too.


Germany has armed cops and a lot of hunters, and fairly high gun ownership rate for Western Europe. Still police practices a policy of deescalation, rarely fires their guns and basically never kills anybody.

Proper training of police (years, not weeks) and a policy of deescalation have at least as big an effect as rates of gun ownership


Regarding Switzerland, IIRC, since 2007, reserve/militia soldiers can keep their guns at home, but not ammunition.


Criminals in Europe also have guns, anyone can trivially fedex these over from the US and you’ll maybe lose 1-2% of the guns you ship.

I’m not convinced that someone that’s willing to shoot a police officer would be unwilling to smuggle weapons internationally.

Guns also get stolen all the time in Europe, yet we still manage.


This got me searching, and it appears the US is the only country where there are more civilian-owned firearms than there are civilians.[1]

It seems reasonable then that this might weigh in as an occupation stressor for police officers.

Especially considering unregistered firearms outnumber registered firearms by 392:1.

In 2013 ... there were 33,636 deaths due to "injury by firearms" (10.6 deaths per 100,000 persons). These deaths consisted of 11,208 homicides, 21,175 suicides.[1]

Approximately 1.4 million people have died from firearms in the U.S. between 1968 and 2011.[2]

That’s approximately ten times the number of soldiers who died in the American War of Independence.[3]

1. https://en.wikipedia.org/wiki/Estimated_number_of_civilian_g...

2. https://en.wikipedia.org/wiki/Gun_violence_in_the_United_Sta...

3. https://en.wikipedia.org/wiki/American_Revolutionary_War


The political power afforded to those who get to make calls about the psychological state of individuals is almost endless. I fully expect social media giants to use their data to classify the mental state of their users for political and social ends. For an idea of the potential for abuse read here.

https://en.wikipedia.org/wiki/Political_abuse_of_psychiatry_...


Reminds me of high-school when a buddy was goofing off in class, taking an online quiz which accompanied a news article about suicide and depression. The quiz asked "Are you depressed?".

A few of us were following what he was doing, and at the end of the quiz he got the result effectively saying seek immediate professional help. Shocked, he exclaimed "What?! I'm having the time of my life!"

Turns out there's an overlap between certain symptoms of depression and being madly in love... and he just got a new GF a week earlier.


> Turns out there's an overlap between certain symptoms of depression and being madly in love

Such as? I'm having a hard time coming up with any.


Two that come to mind are trouble sleeping and lack of concentration, plus maybe social withdrawal (at least from most friendships). But still, seems like a stretch; most depression symptoms are pretty unambiguously negative.


Yeah. I'm a doctor and I've never heard a similar description of symptoms from a depressed individual and an individual in love.

My guess is the wording in the quiz was poor.


I can't recall the details, but it were things like lack of appetite, lack of sleep, unfocused or lack of attention.

I recall the that the questions made sense given the goal of the quiz, but the questions were probably too broad and, which was kinda my point, they lacked control questions to rule out other causes for those effects.


Anyone else think this is Super Creepy? Not long a go I would have expected to find this article from Onion News.

So now platform that arguably actively participates in making people depressed has launched an human assisted AI to report people to authorities. How Orwellian is that.


No, I do not think that a company reporting suicide attempts or messages that appear to indicate an immediate desire for suicide is creepy. This is not just a thing Facebook does, either.

This is not "Orwellian". And the keyword in the claim about Facebook making people depressed is "arguable".


As a black man in America, I really don’t want Facebook sending police to my home for any reason whatsoever.


Perhaps you feel a bit depressed sir? Don't worry, help is on its way http://oi64.tinypic.com/97nokm.jpg


Is the ask here for Facebook not to do anything to intervene?

Times was giving Facebook crap that Facebook Live promoted/publicized suicides, so doesn’t quite compute that their attempt to do something about it is also bad.


> Times was giving Facebook crap that Facebook Live promoted/publicized suicides, so doesn’t quite compute that their attempt to do something about it is also bad.

If Facebook's algorithms were promoting suicide-related posts, then doing something about it might be good, but it does not follow that they should also take an active role in suicide prevention.

You are creating a false dichotomy, both things can be bad at the same time. E.g.: being a thief is bad, being a vigilante is also bad.


Times just likes to bash their competition, Facebook, at every opportunity. They're not attempting to hide their agenda.


> Mason Marks ... argues that Facebook’s suicide risk scoring software, along with its calls to the police that may lead to mandatory psychiatric evaluations, constitutes the practice of medicine. He says government agencies should regulate the program, requiring Facebook to produce safety and effectiveness evidence.

Safety/effectiveness studies certainly seem like a good idea, though I'd be really hesitant to prohibit Facebook from doing this, at least not from worry about false positives (the privacy implications are a different issue).

Facebook is just the first in a line of common-sense checks: a Facebook algorithm detects that someone seems to be suicidal, this escalates to a Facebook employee who reviews it and in extreme cases escalates to law enforcement, law enforcement gets the information from Facebook and decides how to respond, and may end up pulling in psychiatric medical help, which of course involves trained people making judgement calls.

Facebook's role is to escalate, not to intervene, and even that escalation process is done by a human. It seems very unlikely that it would ever result in a situation where someone gets involuntarily hospitalized because some machine learning algorithm had a bug.


I'm bipolar and whenever I start losing some sleep I retake an online version of the Young Mania Rating Scale. I'm able to tell "this is a sign", but the test is an inventory of how many signs there are and an indication of whether I should call the head shrinker out of regular schedule (which costs money) or just try to hold back on prodromal behaviors.

I'm no longer on facebook, but it would be really useful if something I open daily would tell me "hey dude, you're overtalkative and online at odd hours and maybe you should get it checked" -- but give me the agency to do so. I understand perfectly well that there are mental illness scenarios where the capacity of a person to help himself is lost (making involuntary intervention the humane course of action) but even in mania this is the exception rather than the norm.

I can't think of a model of illness severity where machine learning is able to determine a score but the time course of this score is expected to be discontinuous. Before any noise of escalation without my knowledge, I would expect tools to be developed that would help me realize I'm getting ill before losing agency. Everything I just said works for depression too.


My understanding is that this Facebook program only focuses on cases where involuntary intervention feels justified, e.g. someone publicly saying that they're going to kill themselves.

But I agree that it would be great if there were more tools to help with self-awareness and guide people to get help. I've had plenty of mental health problems in the past, and any source of better self-awareness seems useful, even outside the context of seeking professional help. Still, at least with my Facebook usage, it seems very unlikely that it ever would be able to know when I'm depressed except when I'm already in the process of getting help from a friend.

This sort of thing already exists a little bit. A much tamer variant I've seen is Nintendo games that tell you to take a break when you've been playing for a while. And Google searches involving suicide show the message "You're not alone. Confidential help is available for free." and direct you to the suicide prevention hotline.


Facebook's role is to escalate, not to intervene, and even that escalation process is done by a human. It seems very unlikely that it would ever result in a situation where someone gets involuntarily hospitalized because some machine learning algorithm had a bug.

Per TFA they don't actually track the outcomes of their calls because that would be too much of a privacy violation. I'd expect the resulting lack of feedback to be problematic.

Also, I'd say that escalating to the people authorized to use force is a form of intervening.


Is this a 'we spy on them for their own protection' sort of argument? I think there should be extreme privacy protections against profiling like this. Its potentially a great service, but not from Facebook who might sell the info to recruiters 10 years later or some other horrible breach of trust. There needs to be oversight and rules


Agreed that privacy is a real concern with a system like this, though my comment was specifically focusing on the "worry of false positives" concern. I certainly don't claim that Facebook is in the clear from a privacy standpoint, I'm just digging into one detail.


>It seems very unlikely that it would ever result in a situation where someone gets involuntarily hospitalized because some machine learning algorithm had a bug.

Perhaps, but people being involuntarily detained because people misunderstand communications those users thought to be private seems much more likely. I have a group of friends who share memes about suicide and make jokes about it constantly. This is actually pretty common. If some uninitiated outsider read some of my DMs, I’d expect some people I know to be on their way to hospital in handcuffs.

More importantly, why does Facebook think this is their job, and why would anybody else agree with them about it? The idea of the postal service sending the police to my house because they were concerned about the contents of a letter I wrote is horrifying, or the phone company because they were worried about a call I had. I’m not any less horrified about facebook doing this, especially considering how much less I trust them than the postal service or Verizon.


Law enforcement would be forced to act if Facebook informed them.

They would like to cover their asses in case something bad happens.

If they didn't act, they would be susceptible to lawsuits.


That's fair, I guess in my ideal world, law enforcement would just act on the factual information rather than being biased by the fact that a Facebook person reported it as a concern. But as evidenced by things like swatting, police are certainly not the best at responding to reports in a calm and unbiased way. I guess my hope is that law enforcement can learn to appropriately interpret and respond to these reports (including deeming that it's actually not a concern), but maybe that's unrealistic.


  they would be susceptible to lawsuits
No, they wouldn't, as case law is totally consistent about. Police have no duty to act on the safety of anyone except those in custody.


Taking people by implicit threat of force because they've been marked by some global surveillance system is bad.


Oh. my.

Please do not play with people's lives based on an algorithm whose classification output we have no way of explaining.


Besides the obvious question of 'does it work' I see a myriad of other problems with this. Does Facebook have the internal procedures in place to prevent abuse? Will this data ultimately be used to advertise things like anti-depressants at people? I already get ads for "mindfulness apps" and other BS cures (I dont doubt the principle technique as such, but these things are pretty useless and just techies cashing in on a fad IMHO).


Slightly off-topic, but mindfulness approaches have been shown to improve depression and are an important part of CBT/DBT used by therapists everywhere


I don't want to improve depression, I want to get rid of it! :)


This story illustrates why New York Times has fixated on aggressive negative coverage of Facebook. Suicide prevention is another area where Facebook is wiping the floor with newspapers. As an information channel, newspapers can't compete in over most of the landscape. The article includes the head of a premier suicide prevention program, John Draper, praising Facebook's initiative.

Data collection may be bad, but suicide prevention is a beneficial byproduct. Data collection may be bad, but the New York Times won't be accused of leading by example. The article is packaged to ping Google, Amazon, and yes Facebook among it's 32 scripts. It's unlikely any of this there to serve a higher common interest than suicide prevention.

The examples in the coverage show how far Facebook outclasses the New York Times.

  Facebook said its suicide risk scoring system worked
  worldwide in English, Spanish, Portuguese and Arabic 
Facebook is working on something relevant to suicide prevention in Mozambique, Algeria, Dominican Republic, and Australia. Beyond it's role in US public policy, the New York Times will always be approximately irrelevant to people in these places.

It's reporting is parochial. Interviewing two experts in New York, four police forces on the east coast and Ohio. To me as a non-absolutist, judgment about the initiative's tradeoffs ought to consider stories from Brazil, Morocco, Mexico, and Canada. Those people don't enter the parochial conversation at the New York Times.

Preventing suicides is the kind of difference journalism is supposed to make in people's lives. The New York Times is driven of "weapons of mass destruction" reporting on Facebook's suicide prevention initiative because it is so completely outclassed as an instrument for journalistic impact on the lives of individuals.


Facebook the company and platform need to be broken up and the pieces/parts isolated from each other. For all the benefits that people see in using it, Facebook has become the most dangerous online company in the world.


As far as I remember, for every features of Facebook, like private messages, sharing pictures and status, etc, there is at least one alternative, and people are free to move to several competitors. There is no technical reason to force the Facebook company to change their products, but they might have an obligation to educate people about the usages of its website.

You need strong proof to claim "Facebook has become the most dangerous online company". Recently, Facebook groups were used to organize the yellow vests movement in France. For non-techies, Facebook does provide a new way to communicate.

You could always argue "oh yeah, but it already existed 15 years ago with X", but Facebook made it available to the masses.

The security/social concerns happening recently create new opportunities for Facebook's competition, so I wouldn't be that alarmed.

Google, on the other hand...


“Facebook’s rise as a global arbiter of mental distress puts the social network in a tricky position at a time when it is under investigation for privacy lapses by regulators in the United States, Canada and the European Union — as well as facing heightened scrutiny for failing to respond quickly to election interference and ethnic hatred campaigns on its site.”


The New York Times must be the global arbiter of who is the global arbiter of mental distress.


So should we applaud Facebooking for doing this or not?


We'll applaud and curse Facebook simultaneously. They're damned if they do and damned if they don't.


"You haven't logged onto Facebook in over a month. We were concerned so we sent over an over an officer for a welfare check"


“Earlier that day, a local woman wrote a Facebook post saying she was walking home and intended to kill herself when she got there, according to a police report on the case. Facebook called to warn the Police Department about the suicide threat.”

Amazing this is considered bad by the commenters here.


So, you've never said something like "Oh, just shoot me!" in exasperation without actually being suicidal? You don't see how something like that could be wildly misinterpreted in written form, without context, without voice tone, etc?

It's easy to feel like what you say online doesn't really count. People feel that way all the time and that fact often causes problems. Usually, though, those problems don't involve someone calling the cops on you "for your own good." At most, you get banned from some forum.


> So, you've never said something like "Oh, just shoot me!" in exasperation without actually being suicidal

Indeed. Imagine if Facebook called CPS/police every time a user wrote "my dad/mom/partner/boss is gonna kill me when he/she finds out."


> without context

They do have context given how much of a person’s internet life they can “see”. While you can argue that is bad, if they have enough signal a person is likely to do harm and the person says something like that, it seems like it’s a net positive for Facebook to alert someone to help the person out.


I do a helluva lot of commenting online via twitter, various forums and a bunch of blogs. Over the years, I've gotten much more careful about the things I say online on specific topics because people routinely imagine they know me better than they do based on having read my writing for so many years.

So my experience really doesn't fit with what you are saying at all. My experience is that what people say online is the tip of the iceberg of their life and the world imagines it has greater context than it really has and leaps to weird and inaccurate conclusions on a pretty regular basis.

Trying to figure out how to express myself so I feel like I'm actually understood takes an inordinate amount of my time. I have reason to believe most other people don't spend anywhere near as much time and effort as I do on trying to sort that out.

Most of the time, most people don't seem to get as wildly and horrifically misunderstood as I seem to so often end up being. Or if they are misunderstood, it isn't that big of a deal in many cases.

But I remain aghast at the idea of someone calling the cops on me because of something I have said online that people imagine they understood that was not a "threat" against anyone but me.

On top of being skeptical that Facebook understands you as well as you imagine they do, I am also pro right to die.

In a nutshell, if people give a damn about my sorry ass, they could choose to do something meaningful in the here and now to help me make my life work instead of being part of the problem, making my life harder than it already is and then pretending to themselves they are Good People by calling the god-damned cops on me for leaving a fucking courtesy note somewhere online if I decide to finally check the fuck out of here.

Note to self: Don't bother to leave a courtesy note online informing anyone of fuck all. They really don't deserve such courtesies. Geez.


What color is the woman?


Suicide isn’t homicide.


It seems like there's a vendetta against Facebook here. The term "mixed results" is used why? Everything they're doing is completely voluntary. NYT somehow managed to paint Facebook alerting the police about a possible suicide attempt as something negative, just because the police already helped the person. Or about them being too late to report it, again, this isn't Facebook's fault.

You can argue on the term "mixed result", but I think it's reasonable to assume they're indicating that non-success is a negative result, even when Facebook is going entirely above and beyond here to begin with, so there really isn't any "negative result" to be had.

Do people dislike big tech companies so much that reporting immediately concerning messages and suicide attempts to the police must be a bad thing?

This is common practice anyway. If you write that you're about to kill yourself on Reddit and a moderator notices it, I sure hope the police are contacted. Nothing I see to criticize here at all.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: