Didn't realize that was built in to Ansible. Depending on how ephemeral your hosts are, you could scale this to quite a bit of hosts and use host-reported health checks to determine if things are working correctly e.g. HTTP POST a "this went ok" to some web server or metrics/monitoring service
AKA ansible-pull, which can greatly simplify the auth story since the machine only needs outbound access and auth to the repo (if required): https://docs.ansible.com/ansible/2.10/cli/ansible-pull.html