If you use SNI (many domains on one IP), make sure to put this (ssl_protocols TLSv1 TLSv1.1 TLSv1.2;) into the default server { ... } config (or the first one: the one which opens when you go to https://your_server_ip).
The problem is that if you specify ssl_protocols somewhere else rather than default/first server, it won't work. So, yes, serve error, but specify ssl_protocols in there.
With name-based virtual hosts (those that rely on the server selecting the appropriate resource based on the Host header), typical clients depend on the IP address returned by DNS for that host. If they visit that IP, ask for the host, and the server isn't configured to deliver that host's resources, it's good practice to give the client an error. Since the web server has to listen on that IP without knowing which host will be requested before the connection is made, it's convenient to have a fallback and handle errors there. I deny all access to the default host, which generates a 403 Forbidden error (with a custom message), but there are definitely other ways to deal with this situation.
The important thing is that a host's protected resources are served only when SSL/TLS is properly negotiated. Serving one host's content as the default when another host was requested violates this.
In practice, nearly all of these requests come from bots, crawlers and penetration testers. So another advantage is that the log entries can be used to block further requests at the firewall, freeing resources and even possibly protecting the server from undisclosed vulnerabilities (test this approach carefully to make sure it's appropriate for your site and doesn't subject you to a DoS).