The law isn't entirely settled on the matter, but some rulings so far suggest that the government can compel the surrender of a physical key, but not the disclosure of a combination.
So by that definition since my private key is a lot like a combo to my safe, the government cannot compel me to disclose it? That sound like good news.
Also, I guess I am storing all my keys at STL files that get 3D printed and destroyed upon first use. Hello protection from search warrants!
It's the password required to decrypt the secret key that you would not need to disclose. They'll likely end up with the encrypted secret key, but no way to access it. That's because the password is something you know, and divulging it could mean self-incrimination.
Courts have ruled, for example, that the police can compel a suspect to put their face or finger up to a phone to unlock it through biometric means, but a suspect cannot be compelled to divulge their PIN. That's why using biometrics, while convenient, is a bad idea.
This is also a great example of how you can follow what sounds like logical principles to a completely bonkers end-state. Knowledge is protected but your bodily autonomy is not? Eesh.
