> is that the ansible playbook will be idempotent

This isn't really true. I think you are correct that most of the built-in operations are idempotent but you can also do this with a small library of functions in a shell/python script or whatever you prefer. Most things you want to do on provision are idempotent anyways (install this package, download this file) or are trivial to make so (create this directory).

I would take a real programming language any day for the minor cost of having to handle idempotency myself. It would take a couple of hours to reimplement idempotent primitives to replace the Ansible standard library in just about any language.

In my mind the main value of Ansible is playbooks that others have made for you, but many people avoid these anyways to have full control.

I thinj that it's difficult to keep an idempotent shell script or programming language implementation as clean as Ansible over a long period. I deal with a similar thing at work and the Ansible stuff is still mostly good over the long haul with the weird bits like calling other scripts being obvious. The Bash script provisioner we have is just a mess. It's not that an individual can't write a better Bash or Python script but a team of mixed experience, opinions and skillsets coming and going over 7 years definitely cannot. Our Ansible scripts are about half as old, but I don't think the shell script saw significant decline after hitting an inflection point or anything, they just gradually crept away from pure ideals.

I personally find Ansible's value lies in what it makes difficult.