Unless I am mistaken, it should be possible to configure opensshd to first try using keys, then fall back on UsePAM only if that fails (similar to how sshd will normally fall back on PasswordAuthentication).
If not, it might be possible to configure PAM with some sort of keys module set to 'sufficient', then have PAM fall back on two-factor auth paired with pam_unix.
In fact, I'll see if I can get either of those working a bit later today. Seems like it could be neat.
Yeah but (blind) IP spoofing over the Internet is infeasible nowadays. Maybe 15 years ago when ISN randomization was not the rule (successful attack described in http://web.textfiles.com/hacking/shimomur.txt)
But that's correct, in recent openssh versions, it seems that you can add specific-host-only rules for authentication etc.
If not, it might be possible to configure PAM with some sort of keys module set to 'sufficient', then have PAM fall back on two-factor auth paired with pam_unix.
In fact, I'll see if I can get either of those working a bit later today. Seems like it could be neat.