This is really nice. This code looks like it is single-cluster with multiple-GPU. Are there benchmarks on how Horovod work on multi-cluster multi-GPU setting?

In the next part of the blog series, we will showcase the performance across a multi-cluster of gpus.