Amazon had an interesting system for this: one of the interviewers, the 'bar raiser' gets the final decision, ideally based on what everyone else says (but not necessarily).

That BR has done 100+ interviews, took a lot of additional training, and is empowered to ask a lot of questions to dive into the feedback people are giving about the candidate. So if someone is being too hard, or has a weird bias, the BR can override them.

It's a fascinating system and seems to work okay.