Make systemd bind to that port and let it pass the open fd to your process. See socket-activated services.

http://0pointer.de/blog/projects/socket-activation.html