They don't just endanger their own health, they endanger us all. Some people can be carriers but not be symptomatic for a pathogen, so they can be walking around spreading a disease like measles but not even know it. With high herd immunity this is not much of a problem because almost all of the people with whom they come into contact will be vaccinated and not catch the disease before the carrier's body eventually fights it off, but with low herd immunity they become a major problem.

If humanity has not evolved a natural defense to these diseases after thousands of years it is not going to happen by some anti-vaxer volunteering to be an evolutionary petri dish. Actually, now that I think of it, humanity has already evolved a defense against these diseases -- by evolving a big brain it figured out how to develop vaccines...