The bigger issue is you restrict your hiring pool to the set of people who aren't currently employed somewhere that won't allow that kind of moonlighting, etc.

It works, until your company gets big enough, and then it doesn't work anymore.

One of the main reasons that interviews stick around is because they allow for a much larger applicant pool.

A larger applicant pool, AND because they are a commitment device.

As a candidate, I know that the company is burning roughly the same amount of engineering hours interviewing me as I am burning interviewing them. So it's an honest signal that they are at least somewhat serious.

For comparison, one problem with take-home exercises as a first step, is that (especially good) candidates will wonder whether it's really worth their time when their work might just go into the round file straight away.