Part 3 of a 3 part series...
What a difference a few years make in this business. One day you don't have positions open for the all people you want to hire. And the next you are challenged with finding enough people that are the proper fit for your organization.
But how do you determine the proper fit? How can you recognize the characteristics of a great engineer? And most importantly -- how can you tell the great engineer from the mediocre engineer in just the few hours you will have to meet with them?
The process begins with an influx of résumés. Even in today's recovering tech economy, every open engineering position at a healthy company will generate hundreds, even thousands, of emailed résumés. Each and every one of those résumés must be vetted by hand. Your success rate as the hiring manager or team lead will only be as good as the recruiter or HR director who initially pre-screens that stream of applications. This initial step winnows out perhaps 2 in every 10 incoming résumés and passes only those that are strong potential matches back to the team for review.
Candidates that have the requisite background (advanced CS or engineering degrees and years of development experience in related technologies) can then be individually selected for a phone screen by a member of the team. Productive phone screens will not so much focus on what the résumé says -- it is largely redundant to go over that again on the phone, as well as being rather easy on the candidate -- but rather will consist of a series of questions designed to determine whether or not the candidate is likely to do well during an in-person interview. Poor phone screens turn into a "well, tell me about yourself" type of soft-ball interview. That type of conversation is fine for about 10 minutes -- as a way of gauging personality fit -- but it hardly says much about whether or not a prospective candidate is going to to be able to handle the actual job.
The phone screen should contain real technical questions, each ideally requiring nothing more than a pencil and paper and a few minutes to work through. Straightforward design problems are also good -- they show what technologies the candidate is comfortable with. I tend to avoid simple "definition" questions (e.g., "please define 'stack pointer'"), as those can be hit-or-miss and can incorrectly reward candidates who are good at memorizing textbooks. A more informative result can be reached by asking more involved design questions that would require a bit of textbook, and a bit of hands-on, knowledge to complete.
If the candidate does well during the phone screen, consider giving them a simple "take-home" programming question for them to work on and send back to you. Keep it simple enough to be done in 20 or 30 minutes, but be careful not to pick one that is so common that the candidate can simply search online for the solution. Succinct is good -- try asking them to implement one of the standard <string.h> or java.lang.String library methods. Asking this type of question will save you the trouble of bringing in candidates that talk a good game, but couldn't find gcc with ls in /usr/bin. (I will never cease to be amazed at the number of people who have finished PhD programs in Computer Science without learning how to program a computer.)
The day of the interview should be just that -- one day. At worst, if the candidate can not come in for one long stretch, let them spread it out over two consecutive days. Under no circumstances make them repeatedly come back to meet just one more "important" person one last time. Working out interview loops and arranging the schedules is a huge effort -- but it's not an optional effort. Be sure to give ample credit to the people who make all the scheduling work -- their job may just be the hardest of them all.
Five or six one-hour interview sessions is standard, though you can consider doing one of those hours over lunch. The only "informal" interview session should be the lunch interview, if you even have one. Avoid group interviews -- they make the candidate uncomfortable and it's hard to adjust mid-interview based on the candidate's responses. The interviews should be conducted in a private, quiet conference room with a whiteboard. Be sure to offer them water, coffee, or soda, and an opportunity in between interviews to take a quick break.
Every interview should consist of a series of directed and pointed questions, planned out in advance. Make every minute count -- no small talk -- six hours is barely enough time to make a determination about somebody you considering betting your company on. (Remember, just as one good employee can come up with the brilliant idea that doubles the worth of your company, one bad one can bring everyone around him down.)
Do not expect the candidate to lead the interview -- in fact, do not let them lead. Avoid open ended talking questions that last longer than a few minutes. As the interviewer, you should be prepared with a list of things you intend to discover about the candidate -- and be sure to cover all of them, as you will not have a second chance. It is your responsibility, not the candidate's, to leave the room without doubt one way or the other. After all, you are the one that has to make the decision.
Most candidates are nervous and will take a little while to calm down. That's okay -- don't hold it against them. You were probably nervous, too. It is your job to help calm them down. In fact, your job is to help them do well in the interview. You want them to succeed. After all, you want to be interviewing the next successful hire.
If the candidate is bombing the interview completely then it is good to have a back-up question or two that you can move on to. If you are the last interview of the day then you want them to leave feeling good about themselves. If there are still interviews left in day then you want them to go to the next one with confidence and focus, not looking back at what they may have done wrong. Personally, I'd like everyone to leave my interview thinking they have a shot at the job. But if they don't get it, they can at least go back over the interview and put together what they didn't do well on. Most of the time it is just a lack of experience or specialized knowledge, nothing to get discouraged about.
Everyone on the interview loop needs to coordinate to make sure their questions do not unintentionally repeat. That wastes valuable time and looks unprofessional. It is best if everyone publishes their questions somewhere on a company intranet site. This also gives people a chance to borrow good questions and give feedback on ways they can be improved. Questions can and should springboard off of each other, touching on common themes and patterns. Good candidates will pick up on those themes and will even learn something throughout the day. Also, when coordinating the interview loop, remind everyone what position the candidate is interviewing for. Not every position has the same requirements or needs the same questions.
The questions themselves can range from the descriptive and directed, such questions about past experience -- "your résumé mentions that you reverse engineered java bytecode before, what tools did you use?" -- to simple questions about concepts -- "please explain the difference between a hash table and a tree and when you would chose one over the other" -- to the architectural -- "please describe an object model for a racetrack" -- to brain teasers -- "there are n gnomes in a line, and..." -- to the visionary -- "what feature would implement if you had access to this sort of data" -- to technical -- "design a read/write lock" -- to the practical -- "please code an implementation of strstr()". (Of course, I'm only going to give you a sense of our warm-up questions here -- the actual interview loop is pretty darn hard.)
Each of the questions should be designed to determine some facet about the individual and whether or not they are a great engineer. Most questions should focus on determining their programming ability and intelligence, but a handful of questions designed to get a sense of their vision and their versatility are also useful. Sometimes even great engineers won't be offered the job if the fit isn't right. Most of these things must determined face-to-face -- résumés tell only a small part of the whole story.
In fact, I tend to avoid letting the résumé bias me too much prior to meeting a candidate. Instead, I personally like to focus on applied programming questions that are based on real-world problems I've had to solve. In fact, I will often ask a candidate only a single question. I expect the answer to take the entire hour to solve correctly, and it will be broken down into several different parts. The best questions contain both logic problems (ideally with an "ah-ha!" moment) and some practical implementation.
In fact, coding at the whiteboard is not an optional part of the loop -- every candidate will write code. (For those future candidates that are reading this, please be ready to use the whiteboard. Remember, you love to code, otherwise you wouldn't be interviewing there.) The language the candidate choses is entirely up to them -- I've worked through all the solutions before enough times to know how things are going. When a candidate gets stuck along the way I'm always happy to help. It is all about how well we work together after all.
Real world questions are, in my opinion, the best at determining the relative strength of a candidate. There is no room in real world questions for hand-waving answers. Or at least, it's obvious when the candidate is hand-waving -- you can always ask them to show you what they mean with a dry erase marker. Moreover, real world questions are relevant. And a good real-world problem will have many paths to get to a solution -- and thus will show you in degrees how capable a candidate is. (Did they chose the optimal algorithm there? No? How suboptimal was it? Do they see the difference? Is their syntax correct? How incorrect is it? Did they know they could use a library function there? Did they cover all cases? Did they think about ways that their code might break?) At the end of the day I am trying to answer just one simple question -- do I want this person writing the code that our jobs depend on?
Most companies have a tiered rating or HR level system for their engineers. This is a good way of managing compensation, promotions, and job requirements. While you may expect different answers from a junior engineer candidate than you would from a senior candidate, the basic fact that you are hiring for great engineers should never change. The junior engineer you hire today should be expected to move up the skill ladder over time. Any other type of hire is just wasted opportunity. (It's funny, this is just plain common sense at good software companies -- they couldn't imagine doing it any other way -- yet so many businesses just don't get it, and will actually hire people that they think are never going to get that much better with experience.)
To that end, each interviewer is then tasked with one responsibility -- thumbs up, or thumbs down. Inclined or not inclined. Hire or no hire. Perhaps there are a few modifiers, such as "strongly inclined," but there is no "half hiring" someone. You are either going to make them an offer and pursue the candidate or you are going to let them go. And while it's tempting (and easier) sometimes to be on the fence, it is important to make a decision and put it on the table. More than that, type up your notes on each and every candidate and keep track of them. Writing it down is essential -- you will forget your thoughts on most candidates before you know it.
The team should then meet together for discussion. This meeting should be the next day if possible -- at most two days later. If someone feels passionately about a candidate, let them lead the discussion. While it's not a simple majority rule (the hiring manager is ultimately taking the responsibility), remember that everyone in the room is going to be working with that candidate and should have their voice heard.
What I've found is this -- indecisiveness about a candidate means that the interviewers didn't do a good enough job conducting the interview. If the hard questions were asked, and the candidate was really pushed, then there will be little doubt in the end. If there is doubt, double your effort with the next candidate. Refine your questions. Ask more pointed questions. Really get the answers you need to walk away knowing one way or another if you can bet the company on this person.
And after the decision is made, the candidate should be informed as soon as possible. It's a hard job to tell a candidate bad news, and it's just the beginning of another process if it's good news, but either way it needs to be done quickly. I have the utmost respect for the recruiters of the world that do this part of their job well.
While those are all good practices, here are some to avoid: First, don't ever actively solicit résumés, or worse, bring a candidate in for an interview, if you don't have a position available. Value your candidate's time as you would your own (or more, if you don't value your own that much) and treat them with respect. To do otherwise is just insulting and wrong. And once you've made a decision, don't leave them hanging. Give them a phone call that very day if you can. Try to make your decision within a week -- if it will take longer, perhaps because you are hiring only one position and have a second candidate coming in -- proactively give the candidate a call and explain what's taking so long. And never keep a candidate waiting in a room by themselves. They are your top priority for the day, after all. In general, deal with a candidate like you already work with them, even if it isn't going to work out. That's not because of some cliché like "you'll never know when you'll meet them again." No, that's just basic human decency.
No matter what -- don't get discouraged. Building the right team for your company is an extraordinary effort. Never lower the bar. You want to know for sure that every single person you've hired is the best, and a good fit for your company. You want to be able to trust everyone on your team -- and the only way to do that is to hire people better than you are yourself.
Does your company conduct interviews along the lines of some of the best practices suggested here? If not, why not? If you've seen some patterns here that don't work, or could work better, please let me know -- I'd love to hear about them. Hiring is particularly important to me right now, and I feel that a company can never do it well enough.
(Read part 1 on why hiring great engineers is important and part 2 for more on what makes a great engineer...)