Take something like a bank loan. If you had a model at a bank which took credit score, income, wealth, and collateral into account, black Americans would have loans rejected at a higher rate than white Americans. Is this model racist? No, this model doesn't even know what race is, all it knows is credit scores, income, wealth, and collateral. Does the fact that black Americans used to be slaves in the US, or were kept out of certain housing markets, contribute towards the fact that black Americans, on average, have lower credit scores, income, wealth, and collateral? Of course. But is this model racist? Literally not at all. It is completely unbiased, and exactly what the model should be. If the case you're making is that you think that there should be a national effort to correct for historical injustices that were done by the state by actively discriminating by race, that is a completely different discussion.

Having all of our decision-making apparatuses factor in the infinite pile of historical injustices that may have contributed to an individual's particular circumstances is not the way to go. Keep models simple and limited to what is relevant for that particular criteria. Fix injustices further upstream, or you make the whole system a convoluted nightmare.

That is what proponents of the structural racism model are doing. Here's an example I took from the book Weapons of Math Destruction:

When people are convicted of a crime, they undergo a number of personality tests, including the LSI-R (Level of Service Inventory - Revised). This is a highly detailed questionnaire that asks about prior convictions, whether the prisoner had accomplices in their crimes, whether drugs or alcohol were involved, etc.

It does not ask about race.

What it does ask about are things which highly correlate with race, such as the number of police encounters (no criminal suspicion necessary), the number of friends/family/neighbours who have committed crimes, etc. If two first-time offenders have committed identical crimes but one of them grew up in wealthy suburbs and the other grew up in the rough inner city, they will receive very different scores on the LSI-R.

So what do they use the LSI-R for? They feed it into a model which assigns the offender a recidivism risk score. Then they use that risk factor directly when determining the person's sentence, restrictions, parole eligibility, etc.

So now we're not even talking about historical injustices, we're talking about ongoing injustice based on historical injustice. It's a vicious cycle, or a negative feedback loop, if you will. This is a serious problem!

Edit: Just to add another piece of the puzzle, the reason wealthy suburbs vs rough inner cities correlate so highly with race is a direct result of the historical racist practices of redlining [1] and white flight [2]. Now combine that with grinding poverty (also a result of redlining and segregation) and the war on drugs, and the result is high-crime neighbourhoods in the inner city. Those high crime neighbourhoods attract highly increased police presence, which leads to more convictions, which leads to more patrols, etc. This is another vicious cycle which feeds into the above statistical model.

[1] https://en.wikipedia.org/wiki/Redlining

[2] https://en.wikipedia.org/wiki/White_flight

Your indicator for whether or not a model is racist cannot simply be that the model produces outputs that are delineated by race in such a manner that is unpalatable. So long as the model is actually not using race as a means of predicting outcomes, though, any behavior that is racist would simply be due to including poor features.

People commit more crimes when they have less opportunity. People have less opportunity when they grow up in high crime neighbourhoods. This is a negative feedback loop which was started by slavery and accelerated by segregation and redlining.

It’s not enough to use a hands-off approach. To correct the problem requires an active push in the opposite direction, to restore opportunity and break the cycles.

Edit: Think of it this way. You and some friends are playing Monopoly, drinking a few beers and having a great time. An hour and a half into the game (we all know games of Monopoly can last 4 hours or more), you discover one of your friends has been cheating. Now what?

He says "Sorry everyone! I'll stop cheating now and everything will be fine."

Is that true? Of course not. The proceeds from cheating may have been used to acquire the orange properties and maybe even put houses up on them. Every time you and the other friends land on those properties you end up paying rent to the previously cheating friend. Rent that he should not be collecting because those assets were acquired by cheating.

This is what it's like to have historical injustices continue to perpetuate into the future.

This is the problem with processing our world down racial lines. You're trying to correct for a historical injustice. The fact that race factors into the circumstance of why people are where they are right now doesn't change the fact that those variables lead to recidivism. It's not racist. It's accurate.

If you want to fix the problem, then you need to fix the underlying issues, which tend to be economic. Those economic issues stem from an issue that affects all races, and therefore splitting it across racial lines only serves to reduce the possibility of actual change.

All you're doing when you try to account for historical injustice is slapping a band-aid on a deeper issue.

(Edit: Grammar)

That's why the racial angle is a waste of everyone's time and energy. It's not the relevant issue. The more relevant issue is how we deal with crime prevention. Currently, we go with a punishment approach rather than a truly rehabilitative one. This also has a lot to do with economics, and lobbying and private prisons and so on. It's much more complicated than 'everybody's racist'.

Besides that, there are plenty of people around today who actually are racist and they are major proponents of punishment-based approaches. If you try to switch to rehabilitation and intervention, they will resist you. They hold views that some races are innately of lower intelligence and have higher criminal tendencies. You aren't going to counteract that pressure by saying "the racial angle is a waste of everyone's time."

That's only true only if you believe the majority of people are racist. I don't.

There's also a number of ways that you don't even have to interact with that argument. You can show that the end results. It's not like we don't already have the studies and statistics that show how to resolve these issues. You can easily say to someone who thinks that way 'ok sure, you go on believing that, but even if you do believe that, there are still better ways to resolve this.'

Also most people are actually pretty receptive to new information if you are capable of packaging it well, and acknowledging their biases without judging them for it.

But besides this, we have too many people fighting a tug-o-war over the term racist. Some people want to apply the word to everyone. Other people think it should only apply to literal Nazis.

But again, that's a distraction. The membership in the Nazi party was around 8.5 million in 1945, yet the population of Germany was around 90 million at the time. That's right, less than 10% of the population were Nazis, yet they controlled everything.

The topic should be 'How do we fix crime, recidivism, and poverty?'. Pointing out historical injustice does absolutely nothing. There is no reasonable thing to do if that's your focal point, because the only ways to 'resolve' it is what? Reparations? Making decisions about jail time and release based on race? It's illogical. You'd end up putting people with high levels of recidivism back into a community, only serving to repeat the cycle, because you never actually look at the real problem. The things that cause that recidivism. The things that are actually causing it right now, instead of the reasons that it happens to be black people. If history were different, it could have been anybody. It could have been white people, brown people, any color, any ethnicity. It explains why the people in this situation are black more often than they are not, but it does not explain how we fix the problem.

It is an absolute waste of time, and the real issue is that while we screw around talking about pointless grievances people will continue to go to jail, and die, because we're still not talking about the problem.

Edit: I realized that this wasn't quite a response to your last post, but the relevancy is that you will only get complacency if you're focusing on things that can't really be fixed, or that essentially blame others, but what you can do is draw parallels, and essentially say 'Your issues are my issues too, and we can and should work together on them,' which is also true.

In your example, it could be argued that a person who isn't cheating can keep collecting rent on their properties (however "unfair" that might seem) - i.e. the (un)fairness of the current situation (and the degree to which we try to "fix" it) depends on the path used to get there.

In the "inner city kid" example, it doesn't matter how people got there - either due to historical injustices / racism (i.e. "cheating" by the rest of society) or simply because their parents were drunks or criminals or poor or whatever - so, again, race doesn't and shouldn't matter, and helping poor inner city black kids in preference to poor inner city white kids is racism, no other way of putting it.

There was a story recently about NYC cops being given race based targets for arrests. If that data was fed into a system and predictions generated they could be both correct and racist.

That's maybe an extreme example, I think the person you're replying to was trying to illustrate the same thing but with greater indirection between the racism and the arrest.

To give a non-race example, I've heard that ugly people get convicted at a higher rate than good looking people. So a 'hot or not' rating could help predict reoffending conviction rates. I'd assume we would want to adjust our models to avoid that, even though it's not an incorrect prediction.

Fundamentally there's nothing wrong with this. The somewhat harsh truth is that 'systemic bias' is actually just... statistics. There are a lot of minority criminals. It's not that there's something special about these people that makes them criminals - it's the same thing that makes everybody turn to crime: lack of opportunity, low capacity for upward mobility, limited access to education, and so on. We act as if the information is not accurate, and that this is a result of racism, but the cold truth is that if you were to take a white person and a black person and only look at the likelihood of criminal behavior, the black person is going to come out on top of that. It's just the math.

Where the human element comes in is where we decide what to do about that math. Do we blame the race? Do we utilize these models to preemptively police people on the basis of race? The answer obviously, should be no. That said, our model can give us insights into this. We can take this data and go, 'well we know that it's unlikely that race is the major causal factor here, so what else can we look at?'

This is a much deeper issue, and 'structural racism' is a really bad way to look at it, because it forces you to focus on the racial elements, even if they're not relevant. It's asking for a model that is not representative of reality, because it looks ugly, rather than looking at it for what it is - just data - and figuring out what to do with that data.

"Crucially, incorporating more proximal and predictive variables into models, rather than relying on race variables to act as proxies, will improve transportability of algorithms across contexts."

If we want better models then they need to also model structural racism.

The article addresses this

> When “race-neutral” approaches are employed in model development, prediction will tend to be poorer for racial minority populations.... Two explanations for differentially poorer model performance can be addressed by collecting more data: too few observations of members of racial minority groups and unrepresentative sampling that can differentially limit generalizability. However, an additional cause of algorithmic bias is not well appreciated and cannot be overcome simply by adding more of the same kind of data to a learner....

But you also have to realize that this is always going to be somewhat arbitrary.

What? In what world is this assumption being made? Do we assume that every person was born into a stable household? That every person has the same IQ, the same height, looks, had the same lucky encounters with the right people whose needs intersected with their capabilities? There are countless dimensions along which people are not the same, why would you ever assume that whether or not someone gets a bank loan has taken into account every advantage or disadvantage they have been given?

A bank loan is a business transaction where the likelihood of you being able to pay back the loan at the prescribed interest rate is being determined based off of highly predictive features, that's it.

If you want to do corrective social justice, do it in a handful of places, and let the rest of the system operate off of sensible rules. Social justice cannot permeate every single decision made in our society, it is irreducibly complex even on a single decision.

That being said what is unfortunately NOT shocking, is that anyone upvoted this comment at all and it that doesn't have a negative score.

Pretending something doesn't exist (or ignoring the fact that it does exist), and modeling systems under that pretense, doesn't make the thing not exist - it only reinforces the existence of the thing.

I can understand the parent commenter downvoting, esp if not convinced by the position of the article or the subsequent comments.

I knew better than to wade into this topic on this site given the audience demo of this site, I am hardly surprised at the reaction by lurkers.

I am encouraged and heartened by the commenters that actually read the article and have provided excellent reasons why you would want to build systems and models that account for systemic racism and bias.

Very illuminating all around.

Isn't the argument being made in various different place that race is there in the data regardless of whether or not you encode it as a feature because the humans that create the data already used race as part of creating the data for the model. And by creating the data, I mean the interactions in real life that create the inputs.

You can't escape it because it's already in the inputs to the model because it's a rather insidious part of our society.

Generally speaking..

Ignoring or dismissing relevance of race is privilege for those in the majority who can trivialize something that doesn't apply to them. Apply that to models too and see how much they miss. Value can be where others might not.

It's kind of like making a decision that if one group hasn't had an experience of race, it doesn't mean anyone else could have either. It also signals that if they don't see value or understanding in it, therefore there can't be any in it.

It takes a truly open mind to entertain any viewpoint that isn't immediately their own.

The reality is, many people live with a reality that might seem unimaginable to viewpoints like the above.

I practice each day as if humanity is one family. I go out of my way to talk to every kind of person that doesn't look like me. It doesn't mean strangers talk to me, especially from the majority..

When programmers all think and look the same and grew up in the same way and places, software tends catch fewer edge cases of everyone who doesn't look like them, in areas of computer vision.

If we step beyond CV, and look into hilarious things like automatic motion sensor sinks only detecting certain shades (or lack thereof) of skin.

Just some food for thought, happy to chat offline too :)

Upstream may think the model--which to them looks like a black box--is perfectly rational, optimally profitable and socially beneficial in a way that is it is not. We have numerous examples where a computer has driven a decision and humans carried out its orders in ways that harmed people. Remember the man violently dragged off the United Airlines flight in 2017[1]? ICE justifies detention by tweaking their risk management software to always recommend detention[2].

This is why we need to care as people building the systems that make these decisions.

1. https://www.nbcnews.com/storyline/airplane-mode/united-fiasc...

2. https://www.vice.com/en_us/article/evk3kw/ice-modified-its-r...

Basically, African Americans exhibit much higher heart rate variability, meaning their nervous system is much quicker to react to stimuli (quicker time to fight or flight response, for example) and this still isn’t well understood in the field.

A naive understanding is that racial physiology is just different. And plenty of people will stand by this. However, self reported stress scores offer some insight into the difference.

High stress African Americans with High HRV lived as long as Low stress, low HRV White/Asian Americans. Most likely, the process by which the nervous system regulates itself is heavily influenced by life course events.

Medical science, in my experience, lacks in quantifying these social factors, and too often underplays their significance in determining physiological differences. Humans are incredibly dynamic systems, and the case can be made that we adapt to stimuli in order to survive. It’s certainly possible that the physiological difference we observe in different racial populations is due to survival based on this principal.

It’s only recently that I’ve seen research trying to get at these social/physiological mechanisms, but as far as funding is concerned, hard biological sciences are more interesting. Everyone just wants to edit the genome and call it a day, but I think we could get much further if we understood how life events lead to physiological ailments later in life.

Putting your head in the sand and trying to deny the existence of potentially uncomfortable facts actually fuels these fringe thinkers more imo. Part of their whole schtick is that the truth is being hidden from them.

Look at how the media handled the claim that Serena Williams couldn't beat a top 10 male player. Instead of actually putting it to the test, the whole angle was about how insulting and preposterous that was etc etc.

We are not all the same, but we deserve to be treated so. It's as simple as that. Trying to halt scientific progress because it doesn't fit your world view is quasi religious.

If you live in a neighborhood where gun violence is a common occurrence, do you still get startled when you hear gunshots? If so, this would have a physiological effect, because it breaks your rhythm in an abrupt way, and does so at the frequency of hearing gun shots.

Obviously this is highly theoretical, but if true it would mean that just being in the proximity of violence puts you at an extra health risk, one that very few people would assume.

e.g. Hearing gunshots in the distance, you recognize they are gunshots, take note, but continue on your way. Hearing gunshots on your block or within a group you are in, you take immediate notice and react accordingly. But in either case, any time you hear them, is a reminder that the underlying fear never leaves.

It is not an interesting result to say models not modeling reality are less accurate; the cogent discussion is to what degree systemic racism exists IN reality. This is textbook begging the question.

> Acknowledgements: Conflict of Interest: None declared.

> Funding: Whitney R. Robinson is supported by the National Institute of Minority Health

I am not familiar with standards of conflict declaration, but this looks like a pretty clear conflict of interest to me.

https://www.npr.org/sections/goatsandsoda/2017/05/28/5302041...

1. If you have two people with identical relevant behavior and different races, you want the model to score them identically.

2. Each race should receive a comparable distribution of scores.

3. The scores should be as accurate of a predictor of ground facts as possible.

Relax the first desiderata and your model is now either explicitly or implicitly (via irrelevant proxy variables) using race to determine results, opening you up to racial discrimination lawsuits. Relax the second desiderata and your model is now creating disparate impact across racial groups, opening you up to racial discrimination lawsuit. Relax the third and you're leaving accuracy, and thus money, on the table.

It is these cultural differences which cause most of the group disparity in America, not "structural racism", yet the "critical race theorists" (race hustlers and grievance mongers) and their followers ignore these major factors and replace them with straw men.

In order to fix a problem, it's important to understand the actual causes. The sociologists and other assorted race hustlers will only divide us and lead us astray.

> Structural racism refers to “the totality of ways in which societies foster [racial] discrimination, via mutually reinforcing [inequitable] systems...(e.g., in housing, education, employment, earnings, benefits, credit, media, health care, criminal justice, etc.) that in turn reinforce discriminatory beliefs, values, and distribution of resources,” reflected in history, culture, and interconnected institutions (Bailey and others, 2017).

I think I might be misunderstanding, but given this includes “culture”, is this so sufficiently broad such that hypothetical scenarios such as this (no idea if this is accurate) would be captured “white people are culturally more likely to use crystal meth than other racial groups, ergo they are victims of (a certain kind of) systemic racism”?

It seems like this is just a catchall for any kind of error associated with a racial group, and the article is merely cautioning against such errors. If so, it begs the questions “why not just say so?” and “why use such a loaded term like systemic racism?”.

Suppose you are designing a facial recognition system for police to use in the field while investigating a recent crime to see if anyone with a criminal history is nearby.

(Data taken from: https://en.wikipedia.org/wiki/Incarceration_in_the_United_St...) Because blacks are over-represented in the US criminal justice system (40% of the prison population vs 13% of the population) and because part of what defines "black" is the outward appearance of certain facial features, a facial-recognition algorithm which is trained to recognize criminals, with a cost function based on prediction accuracy alone, and facial features as input parameters is likely going to have false positives that over-represent blacks.

It's very important to consider this when you develop a training set. The developer error (who mostly failed to understand Baye's theorem here) might work something like this: They take 100 innocent people's faces at random. (On average it will have only 13 blacks) Then take 100 random criminal faces from inmates. (On average it will have 40 blacks.)

Then mix up the groups into your training set and assign a prediction score 1 or 0 depending on whether or not your classifier has correctly predicted whether or not a face was in the criminal group. Then, based on no other feature than race, your neural net can get better performance based solely on guessing more often that black people are criminals. That's not a good thing. In fact, if it's looking at a black face from its training set, the odds are nearly 2 to 1 that it's one of the criminals, even though the odds that are at least 2 to 1 against a random black person having a criminal history.

The likelihood of being falsely identified as having a criminal history is much greater based on the only variable of being black. And this type of thing has happened several times already in production systems!

Conversely, the same system, trained on the same data in the same training set, can get higher performance than random by simply guessing that any non-hispanic white person does NOT have a criminal history.

Thus, it's pretty important to correct your training set to reflect the correct Bayesian prior, and the underlying structures that sometimes go by the label "structural racism" or "institutional racism" are essentially exactly that reality in this case.

To be honest, I don't think our understanding of these systems is mature enough for us to be just throwing them out into our societal systems right now. There needs to be a lot more testing because the possibility of unethical results is pretty damn high.

That said, I still think that the concept of structural racism is a bad way to look at this problem, as it's simply one form of a common error when looking at sample sets.