Very cool. Next step would be to emulate lighting in the target scene, but that probably requires pose detection and facial landmarks for accurate shading.
Very good idea. And change in real-time if the background is dynamic. And allow the user to set styles such as warming the FG subject, shimmering as if there is a fire or candlelight in the room, etc.