The problem is that manufacturers like ATI or NVidia feel safer legally if they release no source code and do not publish specifications for their hardware. By not doing so, a nosy competitor can't possibly notice anything in NV's hardware design or their source code that infringes on a patent owned by the competitor. If such a nosy competitor did notice something, NV would end up in court over it at significant expense, and would probably have to end up cross-licensing their own patents just for a protection deal rather than for royalties.
It's simply easier legally to keep the source closed and the specs secret. The bonus is that adopting such a policy doesn't hurt sales one bit, because consumers have shown with their wallets that they really just don't want things like driver source code or programming manuals. It's a win-win for the companies involved.
Also, remember when Matrox and 3Dfx were all about open source and NV/ATI were the nasty closed source folks? All it takes is to do a "where are they now" to see what policy a conservative company would take. Nobody got rich open sourcing their drivers and publishing their specs, and at least two companies sank or nearly sank as a correlative effect (not necessarily caused by their openness). The strategic approach is to do whatever the companies that sank didn't do, so as a result NV and ATI are as closed as can be, with the exception of ATI providing some documentation under strict NDA.
The only way to change this is to hit opencores.org and start hacking on your FPGAs. Open, extensible, capable video hardware would go a long way towards humbling the secretive industry players. Once it's designed and mockups are working, real ASIC hardware can be fabbed in quantity. The FPGA gate configuration can be made freely available with the caveat that anyone who manufactures hardware using it and distributes the hardware must pass along the "source code" for the hardware, in the form of the gate configuration as well as the host-visible registers. This wouldn't exactly fit under software licenses, but maybe some contract can be drawn up to handle it.