I would even go farther and include the expected public key directly in the client. That would even defeat attacks where the connection is already compromised the first time you run the client.