Not quite, you're forgetting the T in HFT. If everyone was just buying securities a microsecond vs a millisecond might not matter much. What happens is a lot of HFTs act as market makers. They arbitrage positions they create which screws over any non-HFTs. It also moves the price of securities for no reason which affects everyone holding that security. There's also been plenty of cases of HFTs doing front-running and spoofing.
The effects of HFTs aren't limited to HFT vs traditional traders. Their artificial arbitrage affects people's retirement funds and the like. It wouldn't be so bad if they were just making money off other traders but instead they're making money off everyone with money in the markets.
> It also moves the price of securities for no reason which affects everyone holding that security.
Huh? It's narrowing bid-ask spreads, which is beneficial to other market actors. There's nothing "artificial" in it other than betting that the price fundamentals won't move against them, like any other market maker.
The effects of HFTs aren't limited to HFT vs traditional traders. Their artificial arbitrage affects people's retirement funds and the like. It wouldn't be so bad if they were just making money off other traders but instead they're making money off everyone with money in the markets.