Aug 25, 2009

Some “Developers” Just Can’t Develop

I’ve been interviewing candidates for developer positions for over a decade now.  In my early years I was very much hit and miss in terms of finding the right people and it wasn’t until a horror hire that I realised what I was doing wrong.

Previously, and like so many other people I’ve seen, I would interview candidates by looking at their resumes, asking questions about their history and experience and basically trying to get a feel for the person and how they’d fit in the team assuming that if they could talk about programming reasonably well then they must be able to do it as well.  Then, assuming that conversation left me with a positive feeling, I would do the reference checks and make a go/no-go decision.

This resulted in a success rate of maybe 60%, though I may be flattering myself here. But 60% isn’t great especially when you consider the costs of a bad hire.  Even so, the bad hires I made I wouldn’t call shockingly bad, it’s just that they weren’t up to the standard I was aiming for.  At least that’s what I thought until I made a really, really bad hire.

This guy was special.  He was listed as being a .NET team lead, he talked about his history and skills really well, he was engaging and confident, and his references came back glowing with praise so I decided to offer him the job.  This he duly accepted and started shortly thereafter.  Now one of the things I was doing with new hires at the time was getting them to work on bugs – it helped me knock down the backlog and gave them exposure to a variety of areas across the system.  I also gave them plenty of time early on to learn the ropes and get up to speed and it would typically take a few days to get through the first few bugs and start having some ah-ha! moments with the codebase.

So anyway, this particular person starts and is given his first bug.  Quite a simple bug too and yet it took well over a week for him to get it done and even then he needed plenty of help and prodding along the way.  I was also getting complaints from the other developers that this guy was hopeless and a really bad developer which I couldn’t quite believe so I decided to take some time, sit down next to him and do some pairing.

What a shock I got.  It soon became all to apparent that the guy couldn’t program to save himself.  He… literally… couldn’t… code.  At one point I was breaking down the task to fix the bug into very, very small steps and asked him to put in a simple for loop and he just stared at the screen completely blank.  So I asked him what the syntax was for the for loop thinking he was stuck thinking about the larger problem or that he was freaking out with me being next to him growing ever more frustrated.  He couldn’t do it.  He simply couldn’t tell me the syntax.  Not even when intellisense was doing it for him.  How about a while loop?  Nope.  An if statement?  Only just. It was absolutely frightening. How could I get it so wrong?!  I couldn’t believe I’d made such a monumental screw up as to hire this guy.  I also couldn’t believe that the references about him were so great.

Then the slow realisation dawned on me that I’d been lied to. I know – I’m quick off the mark aren’t I? :-)  Imagine someone lying on their resume! The shock of it! Imagine someone getting their friends to pretend to be their former employer and giving them great references. Silly me.

So what was I to do? I remember an article I read around that time from Joel Spolsky that made me slap my forehead.  It basically says if you want to hire a developer, get them to do some code in the interview!  I couldn’t believe I’d been so short sighted as to not do this before.  Gah! Sure, people can lie on their resumes and get their friends to lie for them as well but when you see someone write code, there’s nowhere to hide.  You can either develop or you can’t.

So from that point on I got people to do a debugging exercise.  I figured if the code was there, then all I’m really doing is getting them to read code, understand it and make a few changes.  Pretty much like normal development :-) So I got one of my team to devise a small program that had a number of bugs in it.  You would fix one, uncover another, that sort of thing until you got it working; and all in about 30 lines of code.  When I was given with the code I debugged it first time in a matter of minutes and then used the time it took as a guide to estimate the skill level of the people I was interviewing (with judgement applied based on how nervous they were).  I’ve been using this same test, with various amendments for language changes and so forth for quite a few years now and it’s been a fantastic aid at helping me weed out those who can code and those who can’t.

So just recently I was asked to help another company vet some candidates for a role that didn’t need as high a calibre of person as I normally look for, just a competent one instead, so I figured my debug exercise might prove a little too challenging for the level I was looking for.  I figured that as a change I’d use something simpler try out a variation of Jeff Atwoods FizzBuzz test.  I figured that if the candidate got through that quickly enough then I could always whip out the debug exercise and see how they went with something a little tougher.  Unfortunately I never got past the FizzBuzz exercise since by the time they finished it they’d killed all the time I’d set aside for it (and maybe a little more).  Was I surprised at this? After all it is a pretty simple coding exercise. Well, as it turns out, No. Not really.

If you have a look at Atwood’s post and even the earlier Reg Braithwaite post then in them you’ll see that they both quote this statement “I’ve come to discover that people who struggle to code don’t just struggle on big problems, or even smallish problems (i.e. write a implementation of a linked list). They struggle with tiny problems.”.

How very true.

By doing the fizzbuzz exercise for real, I merely confirmed what I already knew from my experiences with the debugging exercise. So, what’s the lesson here? I think it’s simple. Get your candidates to write some code during the interview.  You don’t need any special tools to do it either – just use a whiteboard and pseudo code if you want to.  But whatever you do, make sure they prove to you that they can actually do the job, not just talk the job.

Remember: There are plenty of developers that just simply can’t develop.  Avoid my mistakes and do what you can to avoid hiring the wrong people.


  1. Yeah I hear that I am always pulled into interview "web programmers" mostly because I know alot about web technologies I always get people on css, says on the resume they have been doing it fro 8+ years and ask them to write a complex rule and they choke. Most don't know how to use css to select an element based on id versus class.

    I think it comes down to something my wife once said to me, she said " you don't think given enough time I couldn't do your job!" to which I replied "No"

    I think people at some point started thinking of programming as some sort of mindless repetative task type syntax and program work bleh.

    I worked with a guy named boris once a long time ago he used to refer to programming as an art form, some guys draw stick figures and some of us paint the mona lisa

  2. I bet you would like to get answers like Bob in The Phenomenon (YouTube) :)

    Nice post and good tip on adding the coding Q in interview.

  3. I recently wrote about some of the things I look for -

    I like your debugging exercise idea. I don't particularly care for white board code exercises (even with pseudo code) since that isn't what we expect people to do in their job. Having someone do a little diagramming on the board might help, though, just to see how they analyze a problem or design an approach. I really like having a pre-built test app (like your debugging one) so they can sit down in front of the IDE and show they can work in it.

  4. You seriously didn't ask for your interviewee to write code before?

    I should've applied. :P

  5. @rwendi LOL! That was many, many years ago :-) You missed the window by a long way :-)

  6. Experienced this myself, very good article.

    For years I'm asking people to read code and explain it afterwards to me - not some hand waving but where-does-the-value-of-this-variable-come-from. Lots struggle, lots are lost after jumping around in 3 classes.

    Sometimes I ask them to write very simple could, lots of them fail.


  7. Getting the candidate to write code is utterly important even if they are applying for a real experience post.
    Many interviewers give basic algorithms to the candidates. One thing to keep in mind here is to treat a fresh graduate and an experience campaigner differently b/c the fresh one most probably has studied the algorithms recently and may ge it right and this may led you to judge the algorithm-building qualities of him wrongly.

  8. I've only ever been asked to code in one interview and I thought it was great. The debugging exercise is a good idea too. My efforts are well documented on the amusing Enterprise FizzBuzz project on Google Code

  9. I took a test for the FBI that was just impossible. Not only impossible to answer some of the questions but impossible to finish. Somehow I passed it. The lesson here might be to provide an impossible test to an applicant and the one that at least gets a few right is your person.

    This next comment is not to knock at you because I think you used your brain to figure it out so I wouldn't say this applies to you, but it takes smart people to hire smart people. The companies that I've consulted for that were just total crap were the ones where the hiring manager was another worthless dud.

  10. Here's the flipside of this:
    I went to work for a company that put me through 3 interviews without ever really checking whether I knew what I was doing. I almost turned them down, but allowed myself to be talked into it, (and the great salary numbers didn't hurt either.)
    It turned out to be the worst job I'd ever taken. Some of the more senior people there had apparently been working on this code for over a year and yet were obviously unfamiliar with some of the basic APIs of the language they were working in.
    I am now working at a company where I did 2 hours in front of a whiteboard to show how I thought, and coded, and am extremely happy. The language is the same, the codebase is roughly the same size, but we need only a tenth of the number of developers that were at the old place. (Although we have roughly the same amount of space to work in ;-)
    I would *never* again work for a company that didn't make me code in the interview.

  11. Some bloggers apparently can't write, either.

  12. Can I do your debugging exercise? Thanks. Nick

    nicolas patrick fox at hotmail

  13. Having similar problems finding developers at work at the moment. We've come up with a (simple) programming test for them which is working quite well. One of the tasks involves finding prime numbers. A lot of the candidates don't know what a prime number is so we've had to write that one the test; I'd have assumed most programmers would know that. But I can kind of accept that. they don't I guess. What I cannot accept is the guy who claimed to have studied prime numbers at university. And still asked us what they were. Despite the definition being written and highlighted on the test sheet.

  14. @MrKWatkins Seriously? They don't know what a prime number is? Didn't they do primary school maths? It makes me want to cry! (and laugh maniacally at the same time)

  15. Oh c'mon, @MrKWatkins and @Richard Banks. The majority of real-world typical developer tasks don't involve knowledge of prime numbers. And the best way to deal with such a situation would be to find a good library that contains an expert algorithm for finding primes (although obviously all such algorithms have limits).

    Do we ask interviewees in other professions to recall prime numbers? Why would it be different for developers? Sure, a common opinion is that development is "applied math fundamentally speaking", but in most applications strings of text are closer to being the predominant form of data than numbers.

  16. I can sort of forgive the prime number thing in that I've met a few good developers who taught themselves, never went to university and haven't had to do anything with a prime number since they ignored their teacher that day at school. And I've met some people who could tell me what a prime was but couldn't program their way out of a wet paper bag. (Sadly most of the candidates we've had so far...)

    What really annoyed me was that if you're actually going to try and claim to my face that you studied prime numbers (not maths by the way; he said that he has specifically studied primes) at university then you had better know what they bloody well are...

  17. The problem with your revolutionary way of thinking is that if you eliminate both the candidates who cannot code AND everybody who doesn't have two years experience in the hot new technology that first appeared two years ago, you may have no one left to consider.

  18. @Anonymous: Agreed that most real world tasks don't involve prime numbers, which is why we've now included the definition; I was just surprised that the vast majority of candidates didn't know what they were, especially given most studied computer sciences at university.

    And yes in the real world I'd just download someone else's code if I really need to find a prime. However we've found that the task is a good one to help judge a candidate, and a good one to discuss with them afterwards. We can talk through their approach, discuss optimisations, etc. To be honest it's probably better if they don't know much about primes; if they came straight in with some complex but highly effiencient way to determine primes I probably wouldn't understand it and definitely wouldn't have much to talk about! 8o)

  19. @fsilber it's hardly revolutionary to want to see if someone can code, and there's nothing in the exercises that involves hot new technologies. Just a simple console app that does something small (unless you count .NET itself as hot and new)

    @anonymous The point isn't wether you need to use primes in real world situations or not, it's just that determining a prime is a good way of proving you can code soomething simple. A bit like writing code to reverse a string without using the inbuilt string or array manuipulation functions in ,NET. Not practical but good for setting up a scenario to test someone's coding skills

  20. You would not believe how common this is. I am posting anonymously to protect his identity, but I know an "experienced consultant" that, even though he can code somewhat, would not pass muster. He gets paid a lot of money for what he does, but needs a lot of hand holding.

    Although his resume looks impressive, I personally don't think he can develop a solution if his life depended on it. On top of that, he teaches other people how to develop software.

    I think that anyone who reads this post should take all the time that they need to ensure that they get the right candidate. Scrutize an impressive resume, because 8 of 10 times it would be embellished. A good candidate would back his experience up quickly and effectively.

  21. I've proposed about the same idea to my management, but until now they haven't used it, so it is nice to see someone else had the same idea and is using it successfully. A difference is that my idea includes setting up VMWare images that contain different IDEs and OSes so that you can also check some other things like actual IDE knowlegde, OS command line knowlegde and something very important for me, actually being able to debug a program in an IDE.

  22. Of one thing I'm certain, hiring a developer is no easy task.

    However, as I see this often, the right developer often gets taken out of the game because he doesn't know language X or Y or simply does not have > Z years experience in it. I've been to interviews where I had all the required skills but lacked something (example: ASP.NET wasn't their main language but they had some code that required maintenance) and, despite having much more experience than any other candidate, I was still turned down (or given a poor offer).

    A real dedicated developer will learn a language in a short amount of time because he enjoys learning. That's what you need to look for. If someone can pickup a 'fresh' and 'hot new' technology as mentioned before and still code properly, why would you go with the guy who codes in Java all his life and knows nothing beyond it?

    I, for one, know a lot of technologies but not many of them very in-depth. It's more than enough to build a mildly sized application but not to the point of knowing internals. If I apply to a Java job, given that I have about a year of experience with it, I will probably never get it. Yet, I have 5 years experience with a variety of other languages (like Actionscript / Flex) and could adjust quite quickly (I know it's just a matter of learning the libraries).

    Also, I absolutely despise job offers that require almost every single language ever made to see the light of day, they just make me sad.

    Please, when you're hiring someone, take into account experience with other languages and real-world scenarios. Good developers can adjust quickly and will be worth the hire once you have a problem that's out of the ordinary or want to build something in a new language.

  23. Great article. Nice to have someone go into detail on a real experience on this one.

    Unfortunately I have only been asked to code as part of the recruitment process twice in my career and one of those times it was an offline, in my own time test (so some potential to cheat).

    I am disappointed if I go for a role and there is no coding test as it makes me concerned about the quality of the rest of the team.

  24. I like how Hashrocket interviews. You have to pair program with various people on the development team over the course of a week. That gives a good indication of how you'll fit with the team technically and culturally.

  25. I was just going to make fun of you for the Joel Spolsky and Jeff Atwood references but then I felt it wouldn't be worth a lot to both of us.

    If you are interviewing for a assistant professor position in the US or Canada you will have many 1 on 1 meetings with multiple people. You will have to give a research presentation and defend yourself as well many schools will ask for a teaching demo (you will teach a class for free).

    So the entire day is talking, people asking questions and the candidate displaying who he is via a research talk and teaching talk. I think if you're going to spend 40k+ a year on someone you should at least interview them for a day. On top of that you need evidence that they will be appropriate. Teaching, Lecturing and Research (and answering questions about research) are great ways to figure out if someone can be a successful professor. What can you do with programming?

    * Fix a bug
    * Get orientation
    * Code problems
    * Fix a tiny opensource bug (you don't need to expose your code-base here).
    * Talk through solutions.

    Here's how a google interview goes:
    * Someone calls you up and asks you a few programming questions. You talk-out-loud your thoughts and try to solve the question in a google-doc (over the phone).
    * Then you go in for face to face interviews.

    This might reduce your interview load, give each potential interviewee an hour over the phone to code something.

  26. No offense to you but those are the type of people who apply for and work jobs writing .NET (and Java and PHP). There are great developers in every technology, but .NET and Java coddle the coder so much that awful, awful developers can scrape by. PHP is so flexible and allows you do whatever the hell you want whether its a good idea or not so crappy developers can make one page sites without too much trouble.

  27. Having been CTO of a software company for close to a decade, hiring qualified coders has always been paramount. I've never given a flip about whether or not the potential hire had any experience at all in the language(s) I use. Instead, I propose numerous problems and ask them to solve them (with real code) "in their language of choice". If they balk, I specify one of the languages on their resume.

    I'm very gentle about it, I let them know that they are free to do searches online to find references as needed, they're sitting at a computer with the screen split so that I can watch what they do. I apologize in advance for what may seem a grilling interview, and I let them know that I simply want to evaluate their relative proficiency.

    It is absolutely shocking to me how many developers turn in a gorgeous, polished resume, touting "enterprise" experience in this and advanced development in that, who are then unable to perform even the simplest of tasks in any of them!

    I would guess that 7/10 people are unable to perform basic tasks they tout robust proficiency in on their resumes! Even the extremely simple stuff baffles a majority: upper case a string, trim whitespace, write a function that multiplies two numbers together, etc.

  28. Wow. Here's a first for me (being obviously outraged at something as trivial as a blog post): I'd write the same about you (if not worse). Not only, would I do that if I were a colleague or you were a recruiter I interviewed with, but come on! I spent 15 minutes on a typo-ridden blog post, followed by some other bunch of idiots commenting on something that's not even the worrying point (that dude who posted about .Net devs vs PHP devs vs whatever else; really? So you think you know everything, right? Good for you).

    Grow up folks.

  29. Makes me want to vomit with rage. People are having trouble with fizzbuzz and getting jobs, but for some reason I can't even get an interview?

  30. What are you complaining... you are just a .Net developer.

  31. Not only is it good for the employer, its good for the employee too. The last thing I want as an employee is to be in a situation over my head, so I try to be as honest as possible in interviews whilst remaining positive. I enjoy having programming tasks to do in the interview so I don't feel like I mislead anyone in the interview.

  32. If you did not ask basic coding questions in an interview then you are to blame, and you are the one who is a poor developer/dev lead. The other guy is not a developer, he's a liar. But you are just a crappy programmer and crappy at interviewing candidates. Experience with .NET is normally a telling sign of how weak someone's programming skills are.

  33. I'm a sysadmin with a little bit of python skills. I was interested how I would do on the FizzBuzz test. It's a shame I didn't time myself but it took me about 8-10 minutes to get the solution in Python. I then checked it against this: , the main difference being I had

    for i in range(1,101):
    if (i%3 == 0 and i%5 == 0):

    instead of:

    for i in range(1, 101):
    if i % 15 == 0:

    I struggle to believe someone going for a Senior development position. Would take longer than me to do this. That's very sad.

  34. I think the more time you spend screening candidates before you are in contact with them, before you even see their resume the better.

    Only twice have I met a decent developer who I didn't meet through industry contacts.

  35. Dear Outraged Anonymous,

    You should improve your own writing before you criticize others.

    > Here's a first for me (being obviously outraged at something as trivial as a blog post)

    My first suggestion would be to work on incorporating some of the text you place inside parentheses into your main text. It's not that difficult:

    "I rarely allow myself to become outraged over something as trivial as a blog post, however, ..."

    > Not only, would I do that if I were a colleague or you were a recruiter I interviewed

    Commas? They don't work like this.

    > I spent 15 minutes on a typo-ridden blog post

    I'm sorry, it took you fifteen minutes to read this? Really?

  36. This reminds me of when I wasnt able to find developers for my site CodeMeh Programming Forums and CodeMeh Programming Challenges

  37. Richard, you must be joking me! My friend emailed me your post and I read it in horror. The fault here is completely with you for not interviewing properly. I have been a software consultant and a team lead for the past 8 years, I have interviewed developers and was interviewed for projects quiet frequently and I have never heard of that mighty interview technique where an interviewer just "asks about programming"... Next time just start talking about the weather, or more to the point, stop waisting time and money and quit your job, because as a team lead, I can only assume that you are one, you have failed since your job is to hire people superior to you and vet them through and through, and not interview them half-assed, accepting incompetents and then bitching about them in the public forum. Learn how to do you job properly and maybe you would not have to waist my precious minutes on reading your trivial commentary on what is obviously your fault.

  38. Can't believe you didn't say, and nobody's asked -- what did you do with Mr. Incompetent? Did you fire him? Find a non-coding slot for him? What?

  39. @David Mr Incompetent was given lots of time, help and training to see if he could improve. He also had plenty of warning over what would happen if he couldn't do the job, but simply lacked the ability to make it happen. I had to let him go at the end of his probation period.

  40. @pjimmy It's true, personal recommendations and contacts are a much better source of hiring people, but it doesn't scale too well :-)

  41. Just to continue from the above comment ... I would like to add that I rarely see a good developer from a .Net community, ask you about anything but the most obvious and you freeze. It is not your fault really, it is simply the fact that you have no idea how your own platform operates since there is no open source ... @Evan Freedman: Who has ever called a CSS typer a developer? Really? Developer? Every time I get to interview a "web developer" I just ask about a couple of data structures and the candidate falls through. Guys, stop waisting time and more importantly stop writing blogs about development, because that causes me to waist my time, and that is bad for our economy, since I am a real developer.

  42. I'm the guy which the Author is talking about.

    I was a *Lead* when I interviewed for this position, and was hired as a *Lead*.

    You ask me to code?, I'm a *Lead* for effing god's sake.

    Ask, me about for loop, while loop, if statement ... oh! did I mention AJAX? High Availability, scalable stuff and all. I know it all. Do you? Do you???? NO! ... I do ... I'm a *LEAD*, I know it all your effing #@#&*@#&*.

    I quit your company, I got a **Manager's** role at your competitor.


  43. @Anonymous ROFL! Nice try :-) Points for style.

  44. @Outraged Anonymous
    Please tell me you aren't complaining about typos since I use UK English (i.e. proper English) versus American English. Please!

  45. @Richard : "No :-/ ... It wasn't about your `typos'"
    xk0der - On behalf of the Outraged Anonymous.
    (I vouch for him :P)


    Kidding apart. I've seen such people. Interviewed such people. Worked with such people. (Horrors) Worked under such people.

    The anonymous post was to bring out teh thought process behind such [PITA] people.

    - xk0der

    PS: I write in both UK & US English, as should be evident from the "colour" & hues of the anonymous post. :)

  46. I have a seen a lot of graduates fail the fizz buzz question. Unfortunately the other people involved in the hiring process still want to hire them. Though at that point I just have no interest in talking to them any more. Finally the rest started listening to me when we went though 3 developers in 6 months because they could not do anything .....

  47. I have a hunch you're astigmatic given the kind of typo in your post. Interchanging imagining and imaging suggest your eyes have difficulty resolving multiple, adjacent, vertical lines as do mine.

  48. Good article, from the other side, having to code help relaxes a developer and opens them up to communication. As an interviewee, I always prefer a 'test' environment because if I get to code, (hopefully on a keyboard as the white board is hard to edit on, is slow, and generally becomes a mess) then I can say what I am thinking at the exact moment I am typing it. It then becomes a collaborative design process with the interviewer and we can really get into it and exchange ideas.

    This is how I like to code in practice as well, in an extreme environment where one can bounce ideas of one another and get immediate feedback.

    Lastly, coders are generally better at expressing themselves on a keyboard than verbally (I close my eyes when I talk which tends to detract some people). I also talk too fast when nervous and the keyboard will usually bail me out.

  49. You didn't say anywhere in your posts that you had your developers participate in the interview. Why is that?

    Normally a developer can tell in 10 minutes or less if someone knows what they are talking about, and if they can actually do it.

    Maybe that was your biggest mistake.

  50. Hm. I'm almost always giving an exercise to create simple desktop/web application with master/detail view and database persistance (it doesn't matter in what technology stack, candidate can use whatever he likes). Quite hard to compare such an exercise to the FizzBuzz test ;-) but the fun begins after the exercise actually when the candidate is asked to extend this simple application with some features with other developer sitting next to him (which he can also ask for help as I'm also interested if the person can ask "good questions").


  51. You want to know how to tell a good programmer from a bad one?


    Good programmers program, bad programmers get certifications.

  52. @Mike

    No, it's not true : it's experience that makes a good developer.

  53. $:/tmp$ cat
    int main(int argc,char** argv) {
    for(int i= 1; i < 100; i++)
    if(i%3 == 0)
    if(i%5 ==0)
    std::cout << "FizzBuzz" << std::endl;
    std::cout << "Fizz" << std::endl;
    std::cout << i << std::endl;
    return 0;

    Am I hired?

  54. Nope. It doesn't produce "Buzz" for multiples of 5. A bit shit really.

  55. You want to know how to tell a good programmer from a bad one?


    Good programmers program, bad programmers get certifications.