It isn't wrong. These things COULD be useful - just like a camera in your shower that warns you if you have a problematic-looking mole on your ass could be. We could do amazing things if not for bad actors.
But bad actors are a much harder problem than computer vision or AI or recommendation engines or just about anything else non-adversarial. It's why discovering climate change 50 years ago hasn't led to a solution since. It's why we can't just educate people out of misconceptions, we have to actively fight against people trying to keep them wrong and make them wronger (and it's why institutions have lost so much trust that this becomes possible).
The only defense against this is virtuous people in power, but our system has done an incredible job - somewhat intentionally - of making sure that they aren't. Investors aren't lining up to support someone who'll make 10% less profit to avoid some horrible externality.
But bad actors are a much harder problem than computer vision or AI or recommendation engines or just about anything else non-adversarial. It's why discovering climate change 50 years ago hasn't led to a solution since. It's why we can't just educate people out of misconceptions, we have to actively fight against people trying to keep them wrong and make them wronger (and it's why institutions have lost so much trust that this becomes possible).
The only defense against this is virtuous people in power, but our system has done an incredible job - somewhat intentionally - of making sure that they aren't. Investors aren't lining up to support someone who'll make 10% less profit to avoid some horrible externality.