You should look into the Cerebras architecture

https://medium.com/@cerebras/cerebras-architecture-deep-dive...

It's a lot closer to what you're thinking, and you can use it for inference today with an API key.

https://cloud.cerebras.ai/?utm_source=homepage