Sorry, are there people who do an interview like this and then just treat the result as a boolean pass/fail?
I mostly do pair-programming interviews. When I do it async and have them write something, we'll at least discuss the code together. So either way, I'm way less interested in whether they finished than how they did it and how they think about it.
And maybe I'm just weird, but I'm not super interested in the "upper limits of a candidate's abilities". If they're using all their cleverness just to write something, neither they nor anybody else will be able to maintain it. Software is a team sport, so I want people who are good collaborators. Indeed, the "upper limits" thing makes me think of Aphyr's series of fictional technical interviews. (Start with the last one and work your way forward.) https://aphyr.com/tags/interviews
I mostly do pair-programming interviews. When I do it async and have them write something, we'll at least discuss the code together. So either way, I'm way less interested in whether they finished than how they did it and how they think about it.
And maybe I'm just weird, but I'm not super interested in the "upper limits of a candidate's abilities". If they're using all their cleverness just to write something, neither they nor anybody else will be able to maintain it. Software is a team sport, so I want people who are good collaborators. Indeed, the "upper limits" thing makes me think of Aphyr's series of fictional technical interviews. (Start with the last one and work your way forward.) https://aphyr.com/tags/interviews