HMM based speech recognition works ok locally today (and has worked for a while as you say) but there is a large difference in word error rate, handling noisy environments etc when compared to SOTA recurrent neural network models. Those don't (yet) run realtime locally on mobile hardware, but we are not far off.

The difference between 8x% and 97% accurate speech recognition in terms of user experience is pretty drastic

I remember Dragon Naturally Speaking. You had to do a lot of training to your specific voice to get it to a level of accuracy I would describe as "not great." You'd have to slow down your speaking to get anything I'd consider acceptable output out of it. Modern systems are much better.