Your logic is at fault, at least regarding atheists: atheists are expected to be tolerant of religious beliefs, but no religious person is expected to be tolerant of atheist (non-) beliefs.
Expected by whom? There were and are "warlike" atheists, who actively fight, mock and even eradicate religious objects and people, and there are, thankfully, a lot of religious people, who tolerate atheists' non-beliefs.
