Skylake-X has already had its die shots examined and the AVX-512 register file, the dominant part of the layout, is something like .5 of a single core, so it's not even going to buy you much area for anything if Intel deleted it, the whining about how it's better spent on extra cores by Linus is totally overblown. Ice Lake has also dramatically improved the per-core frequency tuning for client SKUs to the point AVX-512 is quite viable on my laptop with no serious problems; a single thread doing something isn't going to tank anything. Ice Lake-X almost certainly has 2FMAs instead of the 1FMA of client SKUs however, so it'll be interesting to see what the new power licensing situation is, but this is clearly something they've had in the books to improve.

The problem is that for the workloads that need specialization, you sometimes really need it. You could also delete the vectorized AES units in your Intel machines too and the general purpose performance wouldn't be affected much, but cryptographic performance specifically would tank, and it turns out, that matters a lot in aggregate for many people.

Ultimately there are literally dozens of specialized inactive units on any CPU at any given time that could be "better spent on general purpose units" (which also isn't necessarily true if other architectural choices prevent those units from being utilized effectively). People just like complaining about AVX-512 because it's easily digestible water cooler chat they read about on a blog.

AVX-512 is problematic in multiple ways your comment fails to even touch upon.

It impedes latency-sensitive applications due to forced down-clocking taking place for AVX-intensive instruction streams. The other side effect is that it quickly produces extreme heat that not only forces further down-clocking of the core but also taints neighbouring cores with dissipated heat and prevents them from going into turbo.

Not everything is about performance in laptops.