Here at Stout we screen a lot of technical candidates. Our first step is to filter the resumes. (We have little need for glass window installers despite the fact that they match the keywords "windows" and "installation!") Then we conduct a technical interview to assess a candidate's skill level. For candidates who will be contracting with us directly we like the extra step of doing a coding exercise.
Why not just request a code sample?
We do like to get code samples from candidates, but we rarely get useful ones—and frequently we get nothing at all. Candidates are understandably leery about sharing code that is owned by the companies they've worked for, and sometimes they aren't able or willing to give an anonymized sample.
Sometimes we get code from hobby projects, but that usually isn't deep or up to production quality. Other samples are just too small to glean anything useful from. Very occasionally we get code stolen from the Internet, which is helpful in identifying candidates who are stupid!
Thus, the coding exercise.
A good coding exercise is about more than just coding. It should also be about process, communication and personality / working styles. Our coding exercise follows a process that is common to many of our contracting projects:
It's important to note that the tasks we give them are generic. They are something any programmer can understand and don't require any specific domain knowledge. While we are asking them to spend time on this exercise (typically less than an hour for the analysis and somewhere around an hour for the implementation), we aren't asking them to do free work. The task they are implementing is for the exercise only. We don't use the results.
The exercise must be small enough that it can be comprehended easily and implemented quickly, while still being complex enough to produce meaningful results. Coming up with a good exercise is quite a challenge!
Having programmers implement the popular "FizzBuzz" test would tell us whether they could code, but it would tell us neither how well they deal with real problems, nor whether they can analyze or estimate.
I've administered our coding exercise to a number of programmers over the past few years, and I've seen some interesting results (focusing on the failures here because they are more interesting!):
No coding exercise is perfect and I'm sure we've passed up great candidates who just misunderstood something or were having a bad day. But our exercise gives us invaluable insight into how a programmer might operate on real world projects within our specific process and requirements.
Tips for Companies:
(1) Recognizing that candidates are in short supply, avoid some of the common coding exercise turn-offs.
Don’t expect a candidate to enthusiastically accept a coding exercise before they’ve had an opportunity to meet with the hiring manager. Most candidates won’t undertake a coding exercise unless they are jazzed up about the opportunity.
Integrate the coding exercise into, say, a half day interview that includes short sessions with the hiring team. Candidates are willing to make a finite investment with a company and its interview process. They balk at interview after interview, exercise after exercise, spread out over several weeks.
(2) If you want to use a coding exercise for your company, first figure out what process points you want to cover. For example, if you do a lot of requirements gathering then perhaps you want to give them an underspecified task and have a dialog where you can see how they elicit requirements.
(3) Focus on finding a task that is simple enough that it doesn't require discovery and isn't too specific to your problem domain (unless you require programmers to have pre-existing domain knowledge). Any coding should be specific to your language / platform. If your company uses a practice like Test Driven Design or pair programming, then your coding exercise should too. It may be helpful to provide a starting project or skeleton code that has TODOs to be finished by the programmer.
Tips for Candidates:
(1) Read the instructions. Then read them again. If there is something that isn’t clear, ask for more information. Many coding exercises deliberately leave out vital information to see how candidates address that.
(2) Focus on completing features or sub features instead of bouncing around and completing nothing. I'd rather see a working backend with no UI, or conversely, a working UI with stubbed or mocked backend then have everything be a work in progress. Maybe you display data, but can't yet sort it or edit it.
(3) As you go, leave TODOs about things you still need to do. That will help you not to forget things, but more importantly if you run out of time it shows that you are aware of something that needs to be addressed, not just that you forgot about it entirely.
(4) Aim for production quality, but a little under polished is okay. You should absolutely do things like closing database connections, preventing SQL Injection attacks, or releasing/disposing of resources (as any production quality code should), but not everything needs to be as clean and refactored as full production code. This only applies to a time-limited coding exercise though! If you are turning in a pre-written code sample, I assume you think it's polished and production quality!
(5) Please test your code, or at least run it! If I can't run your sample or it breaks with a single click you didn't anticipate, it doesn't reflect well! Expect your code to be reviewed by developers who know how to break things (like putting text in number fields).
(6) Comments and error handling matter, but too much of either can be as bad as too little. Again, aim for what you consider to be production quality, with TODOs if you are constrained by time.
Whether you give the programmer time by him- or herself to implement the code or you pair program, you will come away with a far better understanding of how the programmer approaches tasks. If, as a candidate, you are lucky enough to be given a paired programming coding exercise, realize that this is a great way to get to know how your team interacts and approaches problems. For both, coding exercises give you better insight into whether or not you've got a good fit!
This is a technical/business article catered to developers, hiring/project managers, and other technical staff looking to improve their skills. Sign up to receive our articles in your email inbox.
Stout Systems is the software consulting and staffing company Fueled by the Most Powerful Technology Available: Human Intelligence®. We were founded in 1993 and are based in Ann Arbor, Michigan. We have clients across the U.S. in domains including engineering, scientific, manufacturing, education, marketing, entertainment, small business and robotics. We provide expert level software, Web and embedded systems development consulting and staffing services along with direct-hire technical recruiting and placements.