If someone else in the "exact same circumstances and starting conditions" implies they're identical down to every single molecule, how is that someone else?
If they're not identical at that level, they wouldn't make the same decisions. Put two almost-perfect clones into two exact copies of the world and a week later they'll be on diverging paths.
So if the argument is not to judge anyone as a person because everyone would act the same if they had the exact same life circumstances and environment, and everything that affects their decisions fits into life circumstances and environment, what else is left that it would be unfair to judge?
I don't buy that for a moment. It presumes people do not have choices.
The difference between a man and an animal is a man has honor. Each of us gets to choose if we are a man or an animal.