March 15

Creating A Better Interview

If you were going to interview for a role as a chef at a restaurant, what would you expect to see? Maybe the restaurant would ask you to make a couple of dishes on their menu or evaluate how you interact and work with your co-chefs? They might even ask about your past experiences with other restaurants you’ve worked at. What you wouldn’t expect is for the restaurant to ask you to perform the latest Benihana Teppanyaki tricks or ask you to explain the chemistry behind molecular gastronomy. It goes against our intuition for what an interview is: a test to predict our future job performance; however, this is exactly what modern software interviews have you do. Let me provide an example. A major component for most software engineering interviews is the live coding exercise. Most of the preparations for this involve solving various problems on online platforms such as LeetCode. Far from making you a better software engineer, these coding problems test your ability to solve logic puzzles rather than your ability to write clean code. They don’t showcase your ability to use frameworks relevant to your job. They don’t evaluate your ability to make accessible user experiences and they definitely don’t test your ability to make architectural decisions that will affect the long-term maintainability and flexibility of your codebase. So where do we go from here? If the current interviewing process is so far from ideal, what are some of the qualities of an effective interview? Ultimately, what we are looking for is a process that has a high probability of predicting job success. Based on some intuition and research, I was able to come up with the following qualities of an effective interview.

Should mimic work done at the company

What frameworks does your company use? What programming languages? What sort of processes does your company use for committing and deploying code? What sorts of problems does your company care about solving? What sorts of discussions do you usually have before beginning work on a new project? Is work usually done alone or is pair-programming the norm? One thing I know for sure, you’re definitely not solving logic puzzles and you’re definitely not sitting writing code while some is actively evaluating you. An interview should mimic not only the type of work done at the company, but it should also mimic how the work is done. The interview should allow the candidate to go through the entire process of what it would be like to work for the company and then be evaluated on their performance through that process. While it might not be feasible to create an exact replica of the work done at the company, getting as close as possible will give you the most accurate signals as to how they will do when they come to work for you. It also comes with an added benefit, the candidate will have a more accurate picture of what the day-to-day work will be like when they join the company.

Should not allow for bias

You would think that a coding interview would be pretty objective, either they got the correct answer or they didn’t, but I’ve seen firsthand how common interviewers can use their own biases to affect their judgement. Especially with a coding interview that tests much more than riddle solving, you’ll be needing to look at many different signals. How well was the candidate able to explain their design decisions? Is the code written in a way that makes it easy to update? Did they write any unit or integration tests? What sort of test coverage do they have? Was the candidate open to feedback from the interviewers? Is the code they wrote accessible and responsive? Obviously, the specific rubric is specific both to the coding challenge as well as the company, but the point is that we should have a structured way of evaluating our candidates.

Should accurately reflect the values of the company

At Amazon, every year, our peers would evaluate how well we demonstrated the leadership principles. In turn, these would correlate with our performance reviews used to determine our future compensation. In other words, our success at Amazon was directly tied to the leadership principles. While your company might not have the same values, each company and industry has some values that they hold in higher regard than others. For example, your company might be making medical devices. In this industry, it’s usually more important to have provably reliable code, rather than the “move fast and break things” culture that’s common for most startups. Understanding and implementing these values will ensure that you hire candidates that best align with your company.

Conclusion

Every company is different. Every company has different technology stacks, uses different programming languages, aspires towards different goals, and faces different problems. Those differences should be properly reflected in the interview process. While the above qualities do not provide the company with exactly what the interview process will be, they should at least be used to evaluate your current interview processes and see how they could be improved. If anyone reading this is in a position to update their own interview process, I hope that you would be able to use these criteria to create better interviews and hire more effectively.