The Incompetent Code Interview

4 November 2000
[22 Mar 2016 additions]

Earlier this year there was a study published [Dunning-Krueger, almost certainly], saying that people incompetent in a field tended to be unable to recognize incompetence, and to overrate their own competence. Dan Egnor then had the idea that this could be used in interviews: give a candidate a sample of bad code and see if they can recognize its quality and tell you what's wrong with it.

I liked this idea a lot. I figured coding tests don't tell you much: they're usually too short to test much, plus most people can't be that creative on short and stressful demand, whereas reading code and criticizing it should be faster and less prone to causing the candidate to freeze in terror. So I wrote a couple of pages of C++ code, with useless comments, inconsistent formatting, dumb C mistakes, dubious class design, and uncommon idioms, inspired by my own experience and by doing the opposite of Effective C++. It was a surprisingly slow and painful process, actually, to get densely bad code which was still somewhat plausible as a real program. (And syntax errors don't count; I'm after coding practice and style, not the ability to be a compiler.)

I've only fully used this in one interview so far, where the candidate took 40 minutes to go through and carefully catch most of the traps. Other candidates either looked otherwise qualified but said they had only used C and not C++, or couldn't pass preliminary screening to convince me it was worth the time to give them my gauntlet. Dan has used it once or twice, but I don't know the results.

[I've since asked Dan how it went at Microsoft and Google; he says he found it inconclusive, as most people would be confused, and maybe pick out some problems, rather than simply itemizing everything wrong as we'd desired. This doesn't match my more limited experience: our one good candidate did go through it, as (if I recall correctly) did our already hired engineers and architects trying their hand at it.]

So what is the code? Ay, there's the rub. Since it's the only code we have for such interviews (enhanced by line-by-line answers put in comments by Dan, which can be easily removed for the candidate) I can't put it up right now and risk spoiling people.