The Take-Home Is Dead: Interviewing Engineers When Everyone Ships a Working Draft

For a decade the take-home did quiet, honest work for us. Send a candidate a small, real-shaped problem, give them a weekend, read what came back. A clean submission told you a lot: that the person could scope a task, write code another human could follow, handle the obvious edge cases, and stop before gold-plating. It was never a perfect signal, but it was a cheap one, and it correlated. That correlation broke last year, and it broke completely. The take-home a strong candidate sends now and the take-home a weak candidate sends now look the same, because neither of them is the signal anymore — the model is.

This isn't a complaint about cheating. Telling candidates "don't use AI" is both unenforceable and, honestly, a bad instruction — I want engineers who use these tools well, because that's the job. The problem is subtler: the artifact we were grading, working code that handles the common cases, is now the part that got cheap. Any candidate with a subscription can produce a submission that passes review at a level that used to take real craft. So a passing take-home no longer separates the people I want to hire from the people I don't. It separates the people who own a subscription from the people who don't, which is everyone.

The code stopped being the evidence

Once you accept that, the interview problem clarifies. We were never actually trying to find out whether a candidate could produce a working draft — we were using the draft as a proxy for the things we couldn't observe directly: judgment, taste, the ability to defend a decision, the instinct for which edge case will actually bite. The draft was a cheap proxy that used to be expensive to fake. Now it's cheap to fake, so it's worthless as a proxy, and we have to measure the underlying thing directly. That's more work for us, not less. The uncomfortable truth is that AI didn't make hiring easier by giving candidates better tools; it deleted our laziest, most reliable shortcut and left us to actually assess judgment on purpose.

We stopped asking "can you produce this?" — everyone can now — and started asking "can you defend this, and would you catch it when it's wrong?"

What we do instead

The loop we run now inverts the old one. We still send a small problem, but we no longer grade the submission — we treat it as the price of admission and spend the actual interview on it. The candidate walks us through a change they'd make to their own code and why; we push on the option they didn't take. We hand them a diff — often one we generated ourselves, deliberately plausible and quietly wrong in a load-bearing way — and ask them to review it as if it were landing in their repo on a Friday. The strong candidates find the bug, but more tellingly, they can say what class of bug it is and what they'd change about the process that let it through. The ones who were leaning on the model to think for them narrate the diff back to us and miss the trap entirely. That single exercise now out-predicts anything the take-home ever told us.

We also stopped rewarding the wrong reflex. A candidate who reaches for a generated answer to a design question the moment it's asked used to look fast; now it reads as the tell that they can't sit in the ambiguity long enough to have an opinion. So we ask questions that don't have a retrievable answer — "here's our actual architecture, here's a constraint we're under, what would you change and what would you refuse to change?" — because the model can generate an approach but it can't tell you which tradeoff this team should be willing to live with for two years. That judgment is exactly the thing we're hiring, and it's the one thing the tools can't hand the candidate mid-interview.

What this costs, and why it's worth it

None of this scales the way a take-home did. You can't grade defend-your-decision asynchronously at 11pm; it takes a senior engineer in the room, paying attention, following up on the answer instead of a rubric. Our loop got more expensive per candidate and we run fewer, later-stage take-homes as a result. But the alternative is worse: keep grading an artifact that no longer distinguishes anyone, and hire on noise. We'd rather spend an experienced hour on the signal that still exists than an automated minute on a signal that's gone. The same shift that made juniors harder to grow and code review harder to do made hiring harder too, for the same reason — the model is genuinely good at the part we used to use as a filter, so the filter has to move to the part it isn't good at.

The honest one-line version is the one I keep coming back to across all of this: cheap generation didn't remove the hard part of engineering, it relocated it. In hiring, it moved from "can this person produce working code?" — a question that's now free to answer and therefore meaningless — to "does this person have judgment I'd trust to own a decision I can't easily reverse?" The take-home is dead because it only ever measured the first question. We buried it and went looking for a way to ask the second one out loud.

If you're rebuilding a hiring loop for the same reasons and want to compare notes on what actually predicts now, I'm happy to trade what we've learned. Reach me at arun@arunjalota.com.