My considered opinion (from the industry) is that its really shocking how under-invested-in the drivers are.
If a fraction of the effort that went into making the card went into making the drivers and maintaining and fixing them, we'd all be massively better off.
The problem may be too much investment in drivers. Modern graphics card drivers are little operating systems unto themselves; they even have built-in compilers. They are incredibly complex, yet written in unsafe languages for maximum performance, with tons of ugly hacks for compatibility.
What we really need is a simplification of graphics APIs so that graphics drivers don't need to be so complex. That may be impossible without convergence in hardware though.
That graphics drivers are a mess internally says more about the insufficient resourcing to do it properly and cleanly and refactor than it does about the really rather trivial and well-understood APIs that the driver implements.
You think OpenGL and DirectX are "trivial"? I'm sorry, but an API that involves passing text strings of complete programs for the driver to compile, link, and asynchronously run on a vector coprocessor is not trivial to implement, not to mention the complex NUMA memory management going on.
The keywords were picked arbitarily, and are probably not very representative, but it seems like at least nVidia is actually investing in driver/software developement.
If a fraction of the effort that went into making the card went into making the drivers and maintaining and fixing them, we'd all be massively better off.