While not perfect, I want to commend the RiseML folks for doing not only an “just out of the box” run in both regular and fp16 mode (for V100), but also adding their own LSTM experiment to the mix. We need third-party benchmarks whenever new hardware or software are being sold by vendors (reminder: I benefit from you buying Google Cloud!).
I hope the authors are able to collect some of the feedback here and update their benchmark and blog post. The question about batch size comparisons is probably the most direct, but like others, I’d encourage a run on 1, 2, 4 and 8 V100s as well.
Thanks for your feedback and your suggestions (and from everybody else)! We'll make sure to gather all of the valuable feedback and run additional experiments. Different batch sizes and a comparison against >1 GPUs is already planned (and partly executed).
So this is a chip that no one outside of Google is going to be able to get a physical copy of ever?
It makes any benchmarks become Google-cloud benchmarks, right?
Edit: I am complaining a bit about the lack of availability but there's also a real point here. If there's no source for TPUs outside of Google, Google Cloud competes only with other cloud providers and with owning physical GPUs - long term, it has no incentive to be anything but little bit more efficient than these however much it's price for producing TPUs declines.
It's going to be a very exciting multi-company arms race -- at minimum, Google, Intel, Nvidia. Microsoft has their FPGAs, Amazon has their rumors. And there are several startups trying to enter the space. I don't think we're looking at stagnating; very much the opposite. It's going to be fantastic for the field.
(I'm saying this with my CMU hat, not my Google hat.)
I would not see things stagnating but it seems like there's a potential for the individual to get cut out of this excitement if each of these entities is keeping it's chips close to it's chest.
The era of the mainframe, with each provider competing with a custom chip, wasn't necessarily beneficial for individual buying computer power.
Industries go through cycles of innovation and concentration. During innovation cycles, many new non-standard products appear with innovative solutions, the entire pie grows really fast. As growth eventually stabilizes, standards become more relevant and consolidation happens, eventually leading to a stagnation that makes the industry ripe for disruption and change again.
If you look at processors, you see that with the early custom processors, followed by some standardization and copy around the IBM S/360, followed by more proprietary innovation around the PC era, resulting finally in the x86, eventually disrupted by the mobile chips, which then consolidated around ARM and so on.
Making it available outside would require hiring more people and doing more work that they don't need right now.
For a while, it's very likely that Google will be the main user of these, so there's still plenty of incentive for it to increase efficiency and reduce costs.
I thought they were going to provide to other cloud providers? I’m also guessing if you’re willing to purchase a lot of them then they’re willing to talk...
It may be that the other cloud providers would then sell them to those individuals. Indeed, the job of entities called "distributors" to buy big lots from manufacturers and break them up.
And of course, I don't know what the point of (apparently) keeping them out of the average person's hand would be.
For starters, Google especially for enterprise level hardware is known to limit their need to provide customer support; imagine supporting the chip would fall on Google, not any 3rd parties.
While not perfect, I want to commend the RiseML folks for doing not only an “just out of the box” run in both regular and fp16 mode (for V100), but also adding their own LSTM experiment to the mix. We need third-party benchmarks whenever new hardware or software are being sold by vendors (reminder: I benefit from you buying Google Cloud!).
I hope the authors are able to collect some of the feedback here and update their benchmark and blog post. The question about batch size comparisons is probably the most direct, but like others, I’d encourage a run on 1, 2, 4 and 8 V100s as well.