(Odd, in all the HTML5 video debates on HN, I hadn't heard that argument from Theora supporters before.) Most of the hardware decoders in smartphones are programmable DSPs; e.g. TI's OMAP3 family. But there still are plenty of chips out there that use fixed-function h.264 decoders, e.g. TI's DaVinci family. The fixed-function ones are smaller, cheaper, and probably lower power.
TI's DaVinci family includes programmable DSP and ARM based solutions and you can buy various 3rd party software implementations of codecs such as H.264 to run on them.