For similar reasons, I ended up using x11vnc as server, and Remmina as the client. Granted, it has been a few years since I last needed it, but this was what worked best, and mostly out of the box too.