Am I missing something - I thought that GPUs are for training the weights

Being able to create an agent in any language to run on any hardware has always been possible hasn't it?