Old-school BT headsets are low-latency enough, afaik. But yeah, just blasting the Opus directly from the network to the headphones would solve it, even re-coding in low-latency configuration only adds 5ms.