Yep, I thought they might have some secret sauce in terms of training techniques, but that doesn't seem to be the case.

"Christian nationalist" is a term that on the one hand, insults Christians, and on the other hand flatters these power hungry grifters. Why would you use such a term?

> they also do not dehumanize large swaths of the population

They dehumanize and kill everyone equally.

I think the West (US especially) has a great system of government figured out, and I wouldn't try to say any group attempting to break that is better than another. It's not disingenuous to say that both are authoritarians who kill millions.

Please don't be childish. I have no love for communism and I get that you probably have some personal grudge against communism because of what your family has gone through. But the way you are expressing your hatred of communism is diminishing the evil committed by the NAZIs and the suffering of their victims, and that's just not right. And it's also not winning you any friends and you'd be better served if you showed solidarity with victims of other evil regimes, instead of trying to up them with your personal victim-hood.

The above commenter's point was actually drawing a distinction between these, and "one-upping" by saying there was a difference, or worse one, out of these extremist ideologies.

I disagreed and said that they are both equally horrid.

Expressing equal hatred for people and systems that kill countless millions through starvation, gulags/concentration camps, and disappearing them, is not childish. I have a ton of solidarity with victims of other evil regimes.

AlexNet was only released in 2011. The progress made in just 14 years has been insane. So while I do agree that we are approaching a "back to the drawing board" era, calling the past 14 years a "failure" is just not right.

If NVIDIA is blocked in China and China keeps developing AI models of comparable quality using home grown chips, it puts into question the dominance of NVIDIA in the market. If NVIDIA is allowed to continue operating in China, it doesn't matter much if China actually uses NVIDIA or Huawei, as long as there is some plausible deniability that China is using NVIDIA powered clusters for their strongest models.

There are a few "optimization startups". But in this context I find it a bit ironic that pretty much everyone is working with the same architecture, and the same hardware for the most part, so actually there isn't really that much demand for bespoke optimizations.

Those that are serious are paying through the nose for their engineers to work on these optimizations. Your competitor working on "the same hardware" does not magically make your MFU go up.

And when you have enough spending to account for 1%+ of revenue for the AI hardware companies?

You can get the engineers from those very hardware companies to do bespoke optimizations for your specific high load use cases. That's something a startup would struggle to match.

> When I look around, I see hundreds of billions of dollars being spent on hardware – GPUs, data centers, and power stations. What I don’t see are people waving large checks at ML infrastructure engineers like me and my team.

That doesn't seem to be the case to me. I guess the author wants to do everything on his own terms and maybe companies aren't interested in that.

There's probably a bit more to it. It really only takes one company to bet on optimizing infrastructure, to the degree that the author suggests to undermine the entire house of cards being built on Nvidia GPUs currently, yet not one AI company is willing to take that bet?

The author could also be correct. Investors tend to be herd animals, and if you're not buying into the same tech as everyone else, your proposal is higher risk. It might very well be easier to say to an investor that you're going to buy a million Nvidia GPUs and stuff them in a datacenter in Texas like everyone else.

I'm interested in the one company that does take the bet on infrastruture optimization. If that works, then a lot of people are going to lose a lot of money really quickly.

Does anyone know why they brand it an "inference chip"? Is it something at the hardware level that makes is unsuitable for training, or is it simply that the toolchain for training is massively more complicated to program?

Very simplified, AI workloads need compute and communications and compute dominates inference, while communications dominate training.

Most start-ups innovate on the compute side, whereas the techno needed for state of the art communications is not common, and very low-level: plenty of analog concerns. The domain is dominated by NVidia and Broadcom today.

This is why digital start-ups tend to focus on inference. They innovate on the pure digital part, which is compute, and tend to use off-the-shelf IPs for communications, so not a differentiator and likely below the leaders.

But in most cases coupling a computation engine marketed for inference with state of the art communications would (in theory) open the way for training too. It's just that doing both together is a very high barrier. It's more practical to start with compute, and if successful there use this to improve the comms part in a second stage. All the more because everyone expects inference to be the biggest market too. So AI start-ups focus on inference first.

They also have the 'tyr 4' [1].

It doesn't have to compete on price 1:1. Ever since Trump took office, the Europeans woke up on their dependence on USA who they no longer regard as a reliable partner. This counts for defense industry, but also for critical infrastructure, including IT. The European alternatives are expected to cost something.

[1] https://vsora.com/products/tyr/

Nothing in principle. But Huang probably doesn't believe in hyper specializing their chips at this stage because it's unlikely that the compute demands of 2035 are something we can predict today. For a counterpoint, Jim Keller took Tenstorrent in the opposite direction. Their chips are also very efficient, but even more general purpose than NVIDIA chips.

How is Tenstorrent h/w more general purpose than NVIDIA chips? TT hardware is only good for matmuls and some elementwise operations, and plain sucks for anything else. Their software is abysmal.

Of course there's the general purpose RISC V CPU controller component but also, each NPU is designed in troikas that have one core reading data in, one core performing the actual kernel work, and the third core forwarding data out.