You could store the decryption key on the disk, only loading it when needed, and possibly byte-by-byte. This is all hackable, especially when such techniques are used mainstream, but it increases the amount of work needed to hack something. In the end it's the OS's responsibility of course.
The .NET framework has a SecureString class which does this. It mitigates the risk of sensitive information being discovered through running 'strings' on memory dumps and the like. It's meant to prevent trivial recovery, not stop an attacker with sufficient time and skill.