Hiring is hard. You're taking a gamble that the person you select has the right skills to be productive, the right personality to mesh well with your team, and the right drive to make a difference. The interview process tries to bring to light as much information as possible about the candidate to help you make informed decisions, but there's only so much that can be extracted in a limited amount of time. Sadly for the traditional developer, interviews frequently devolve into tasks or tests that are, at best, proxies for the actual competencies you're trying to quantify. While development is a logical activity, seldom do developers face questions during the course of their actual job such as, "why are manhole covers round?" or, "what's the minimum number of drops it would take, given two eggs and a 100-story building, to determine the floor at which they'd break?" (c.f. this blog post for a humorous take on this.) Similarly, the typical developer does not produce code by wearing a nice suit, standing in front of a whiteboard and talking for thirty minutes, while drawing unreadable diagrams. (Maybe we'd produce better code if design was argued more, but that's a rant for a different day.)
The benefits of more directly testing the core skills of a developer are obvious: you're getting a much closer picture of their competency with the kinds of tasks you'll actually expect them to do, and from the other side of the table (putting on my engineer hat for a moment instead of the manager one), it's a heck of a lot more fun to write actual code than answer yet again the dreaded questions: "Oh, so you know Java? What's the difference between an abstract class and an interface?"
Some companies incorporate this by including programming take home assignments, but I think this is suboptimal on two levels. First, there is no guarantee that the candidate can't get inappropriate external help. Second, putting on my engineer hat again; respect my time please. If every company wanted me to grind out several hours for coding assignments, in addition to all the mechanics of getting to and from face-to-face engagements while I was looking for work, it doesn't take long before the number of interviews in a given week would suddenly become more than a full-time job. Given the current market for qualified technical staff, any individual seeking a new position will be forced to manage a tsunami of interest, so it is unwise to give them incentives to skip your HR process (like the natural inclination towards avoiding tedious busywork). You should be able to get a good feel for the core applicant/position fit in an hour at most; speaking frankly, approximately 60-70% of the candidates I see are a clear no hire within the first 5-10 minutes and that rises to 85+% by the half hour mark. Why drag it out?
So, it is a good idea to test skills directly, but how should you go about doing it efficiently? Two approaches come in very handy for this, one fairly lightweight and suitable for phone interviews, and one perhaps a bit better reserved for in-depth, face-to-face ones. Each are covered in turn in the next two sections.
Collaborative Text Editing for Phone Screens
The first approach has to do with incorporating collaborative text editing into a phone screen. This does require that the applicant be near a computer (accordingly they should be informed as such beforehand), but "being near a computer" doesn't seem like a high barrier to impose on a software developer ("being away from one" might be a bigger issue). Essentially, it is good practice to include some "FizzBuzz" type coding problems in your interview process to quickly weed out the people who, shall we say, wouldn't find much if you gave them the proverbial flashlight and free use of both hands. The literal "FizzBuzz" is probably too common now and could be regurgitated via rote memorization, but it isn't hard to come up with some sort of coding challenge that a competent person could write a solution to in at most 10 minutes. Any first year CS curriculum that publishes their problem sets would be a good source of questions like these.
The twist is that with the right sort of collaborative editing tools, both the applicant and the interviewer can watch the person write code in real time. This can be surprisingly illuminating. Do they type the solution quickly? Do they sketch out some example outputs for given sample inputs to a function for testing? These are all good signs. Speaking for myself, I don't tend to penalize applicants for minor syntax errors, but it is also a good sign if they produce perfect code, of course (e.g., remembering semicolons, indenting appropriately, etc.). If you ask for something C-like and they forget both braces and semicolons in every instance where they would be required, you can safely raise an eyebrow at any claims of being "detail-oriented" on their resume.
Two free tools that are available and should work across most any platform capable of running a reasonably modern web browser: Google Docs and collabedit.com. The biggest challenge with collabedit, surprisingly enough, has proven to be communicating the URL over the phone for whatever reason, so including it in the initial stage of the process in an email might work better. Creating a new Google Docs word processor document for each applicant is slightly more heavyweight in the sense that it's more than the one button click required for collabedit, but it's also easier to save them in a collection for future reference if need be that way (they'll auto-save so you only have to create them in the collection to begin with). It helps to have problem definitions pre-typed along with any supporting data so that they can be quickly pasted into the editing interface during the interview.
A third option for collaborative text editing that is specific to Unix-based environments is a shared screen or tmux session and a command line editor (e.g., vim, emacs, nano, et al.). This is covered in more detail in the next section.
Unix Tools for Maintaining Interview Environments
Of course, collaborative editing in a text editor doesn't have the same level of fidelity to actual work responsibilities as a full, live working environment. This section addresses that with the goal of getting a candidate as close as possible to the sort of day-to-day environment they'd be using. Although I try to keep many of my process-oriented articles as technology agnostic as possible, this section is written with the assumption that you are a Unix-based shop and that the majority of your work occurs on the command line. This is a natural fit, in the web development space, for most of the LAMP or LAMP-like shops that use some form of scripting language as their main server-side application development platform. Organizations that rely more heavily on IDEs (e.g., companies who have decided to base their work around the Microsoft toolchain) could still use some of the techniques listed here, but would need to adapt them.
It is handy to have a dedicated "interviewee" or "candidate" account. If your company is commonly interviewing many candidates in parallel, these may require numbering (e.g., candidate1 ... candidateN) and scheduling/coordination much like physical conference rooms for an in-person interview. Depending upon your organization's security requirements, these could reside on a shared development machine or on a dedicated host. (Note that, of course, with the prevalence of virtualization technologies, that "machine" and "host" could just as easily be a virtual machine running somewhere on your network.) For the typical interview usage profile, resource needs for a dedicated environment would likely be very modest unless your technology stack happens to require something extraordinarily resource hungry. Even a low-end developer's workstation (as of this writing in 2012) could easily handle spinning up a 32 bit linux install with say half a gigabyte of ram in VirtualBox and be perfectly adequate. (NB: you'd have to set up port forwarding to connect a port externally visible to the running VM to allow ssh access.)
For face-to-face interviews, the practicum environment described above is obviously useful when combined with a computer and projector. Speaking from personal experience, nothing quite so adequately simulates the need to work under pressure in a real environment like writing a solution with an interviewer sitting at each shoulder watching every keystroke. This is easy to pair with the traditional "sketch your design on a whiteboard" approach, allowing you to get a more full picture of the candidate's skills in both design and implementation. It may seem like overkill to have a pre-set interview environment when any developer workstation might do, but it is useful to have a standardized, pre-configured place for candidates to work both to save time (there is no requirement to try to figure out where to have them work or what to work on) and to create a level playing field for all applicants.
For remote interviews (whether for briefer format phone screens or more in-depth second stage interviews that can't be conducted in person), it is possible to use terminal multiplexing tools like GNU screen or tmux to set up collaborative viewing of a terminal session. For GNU screen, the procedure is:
chmod +s /usr/bin/screen(assuming that is the path to your screen binary)chmod 755 /var/run/screen- add the following lines to the candidate account's
.screenrc:-
multiuser on -
acladd {viewing username}(once for each viewing account)
-
- (from the candidate account, e.g. automatically via shell init script on login):
screen -S interview(or some other fixed name, as desired) - (from the allowed observing account)
screen -x {candidate account name}/interview
For tmux, the process is simpler:
- (from the candidate account, e.g. automatically via shell init script on login):
tmux -S /tmp/interview(or other convenient name for the socket, as desired; note that this is not deleted when the program exits) -
sudo tmux -S /tmp/interview attach(optionally, you could alsochmod 777the socket and anyone could do the above without needingsudo)
Note that in both the screen and tmux use cases, the observing user can send input events to the shell as well (e.g., moving the cursor to a given line to highlight a region of code in order to ask questions about it).
Now that the candidate has a place to log in and work during the interview (and a way for their work to be observed by interviewers), it's important to spend a few words talking about what they'll be working with, and on. None of this is vital, but all of it will help make life smoother for both you and your interviewees.
First, as a matter of kindness, the shell environments for the candidate account should be reasonably set up: a complete usable PATH environment variable, reasonable editor configuration, etc. You don't have to go overboard customizing these, but if you're giving an interview testing (for example) perl knowledge, making the candidate find the perl binary on your system or figuring out which one is appropriate ("usr bin? opt nytdbin? usr local bin?") is probably a bit excessive. It is helpful for coordination purposes to set up editors, if possible, to display line numbers for easy reference, e.g. "set nu" and "set ruler" in the interview account's .vimrc. Shell initialization scripts may also be useful to ensure that the working environment is clean (e.g. cleaning out standard account-specific tmp or data directories that could be modified during the course of an interview), either on login or logout as appropriate. Similarly if the interview involves modifying other resources like a database system, restoration to a known good condition as a part of that process would be advisable.
Second, it is extremely useful to keep an interview problem set and all associated working data in source control. This could include program fragments to debug, data files to parse and report upon (e.g., a csv), SQL files to apply to an interview database, and so forth. Retrieval of these resources from source control could be automated as part of the environment setup/cleanup shell initialization script work mentioned above. The source control system used is not terribly important as the likely working set size for interview data is a few tens of files at most and branching/merging/other advanced use cases are not likely.
Conclusion
So, putting all of the pieces together, you'll be able to do things like asking a developer candidate to perform tasks like ssh to a Unix account, use command line tools to debug a program, write a new one, develop a quick and dirty report about a data file, write an ETL script to dump a database table to csv or whip up a quick RESTful web service, and more. Doesn't that sound a bit closer to what you hope to pay them to do every day rather than figuring out how many races you have to conduct with N horses of group size M to find the fastest one?
