RSA has plenty of mathematical gotchas that, if you don't know about them, will lead to a weaker implementation.

If the NSA hasn't demolished public key crypto, it still seems reasonable to assume that they've made sure a significant implementation vulnerability (that looks like normal crypto give and take) has been inserted.