Why Coding Tests Are A Bad Interview Technique

One thing I’ve noticed in hunting for a job recently is the number of companies that insist that you write them a code sample to spec. Not just any code sample, but a fully functional, complete application. This is absurd, for several reasons.

Eli White spends a good deal of time arguing why coding tests are bad. I won’t rehash that here.

My biggest pet peeve with these types of tests is this: there are a lot of companies out there, and I’m sending out resumes to each one that I can find. I simply do not have the time to write fifteen code samples a day, just because you want to evaluate me against your coding test. Period.

Joel Spolsky talks about how it’s important to ask developers to write code during their interview. But he also goes into detail about how he interprets this in his own company: namely, he asks people to write functions during the interview. He doesn’t administer a coding test. He wants to see how people think, how they respond to critique, and how they come up with solutions. He likens the coding test to asking a plumber or an electrician to provide some sort of proof that they’re capable.

But asking a programmer to write a complete application for you is like asking an electrician to build a small electrical grid for you before they work on your house. Can you imagine if every plumber had to prove that they could snake pipes before they could work on someone’s plumbing? Plumbers would be even harder to get to your house than they are today.

Asking a developer to write an application for you before you’ve made a formal offer of employment is also disingenuous, and disrespectful. It’s disingenuous to the value of their time, and disrespectful, especially to mid- and senior-level candidates, because it assumes that they don’t have an understanding of the language and they must prove to you otherwise. A coding sample, in the case of a mid-level developer, or a quick few phone calls, in the case of a senior developer, should establish pretty quickly as to whether or not they’re qualified.

A coding test also doesn’t identify the most important aspect of a programmer: whether or not they’re capable of learning. A fantastic test might represent their place now, but it doesn’t represent their ability to adapt to changing environments. The number of frameworks and projects in PHP should be evidence enough that the language is constantly evolving, and that we’re going to need people who can learn and adapt.

For those who administer coding tests, please reconsider what you’re doing. For junior developers, they’re fabulous tools. But for anyone who has a body of code samples all ready to go, ask for one. Ask a few technical questions that require code in an interview. And make your decision based on how they think, how they learn, and how competent they are.

Brandon Savage is the author of Mastering Object Oriented PHP and Practical Design Patterns in PHP

Sudheer (@bngsudheer) wrote at 11/2/2009 1:08 am:

ou raise a very valid point. It depends on how you evaluate the code written by the candidate. At Binary Vibes, we have made it mandatory for candidates to write code during the interview. In most cases, the programs don’t need to be functional. We’re interested in learning what approach the candidate takes to develop the solution.

The same applies for system administrators as well. Few months ago we interviewed a handful of Red Hat certified engineers. Initially, it was hard to evaluate their knowledge and experience by asking questions. We asked one candidate to install and configure an email server. We provided all the tools needed to perform the task – SSH access to the remote server, preconfigured DNS settings, etc. We allowed the candidate to use Internet for documentation and help. We made it clear to the candidate that there was no need to build an enterprise class mail server. Just a functional mail server capable of receiving and sending emails. After couple of hours, the candidate had managed to edit few lines in the sendmail configuration file. Emails sent to the server were not visible anywhere. I asked the candidate, where the incoming emails stored on the server. The candidate replied that the server stores emails in mutt.

Certification programs like RHCE can be deceitful. I don’t know about Zend PHP certification yet. If you are hiring a software professional, you have to ask them questions and ask them to write code as well. Once you evaluate both, the code and the answers to the questions in the interview, you get a better idea of the candidate’s capabilities.

I do agree that asking the candidates to write full fledged applications isn’t the best thing to do.

Daniel Andre Eikeland (@zegenie) wrote at 11/2/2009 2:56 am:

Completely agree with you, Brandon.

“Certification programs like RHCE can be deceitful. I don’t know about Zend PHP certification yet”

Unfortunately, a certification doesn’t prove anything wrt how competent the developer/sysadmin is – just that he is capable of passing the certification test. It’s like a driving test – if you pass, you get your license – but that absolutely doesn’t mean you’re a good driver.

You can pretty much use the same comparison on developers – you definately need to know how they *think*, how their approach to a problem is, not to mention that you should try to discover if they have any potential beyond the level he/she is currently at.

When I interviewed for my current position, I was required to bring code samples, but more important than that – my interviewer and I discussed different approaches and techniques where they could easily just asked me to write something up instead. In most cases I think you can spot good candidates a lot easier this way – provided of course that you do bring *some* code samples.

When it comes to sysadmins it’s probably a bit different story, but it seems like the tests you mentioned in the comment revealed the lack of knowledge for at least one candidate ;)

Felix wrote at 11/2/2009 3:54 am:

Yes. Absolutely coding test is abused. For junior programmers, this approach applies. But for senior programmers, the most important point is how they solve the problems, or how they think. Unfortunately in China, almost all software companies are doing this. There’re worse cases: many interviewers make fun from asking questions about some unusual technical details!

Bron Gondwana wrote at 11/2/2009 4:19 am:

Oh my goodness – 15 applications per day?

Do you really want to work in every single one of these shops? Imagine everyone doing this? No wonder they want to apply some sort of up front test to separate the people who actually want to work there from the people throwing their resume around and hoping it sticks somewhere.

I have never considered applying for jobs in more than about 3 places at once, having researched them and decided they’d be good places to work. Remember it’s about a relationship that will consume a good part of your life for the time you’re working there – getting a good fit matters more to you than it does to them.

In summary – look at it from the other side and you’ll see why it makes sense, especially with resume slingers like yourself on the market.

Cal Evans (@calevans) wrote at 11/2/2009 4:35 am:

Brandon,
I realize that this post is coming from the frustration of the job hunting process. However, speaking from he other side of the table, the manager doing the hiring, I do agree with you on this point.

I’ve hired more developers than I can remember in my career and tech screened others for companies around the world.

I mean no offense to @Sudheer but requiring a coding exercise just to show how someone thinks is laziness on the part of the hiring manager.

Interviewing a developer is not a process that you can take shortcuts on. Anyone who has been around me for more than 10 minutes knows my thoughts on hiring developers, team interviews and the like so I won’t rehash them here. I will say that the only way to truly asses a developer’s skills is to have him talk with his peers about programming for a while. I find that engaging the potential hire for 15-20 minutes about a given technical topic – Unit Testing, Design patterns, single vs. double quotes, whatever is appropriate for the skill level you are hiring – is really the only way to really assess their skills. If they can’t talk about a topic intelligently to their peers then the certainly can’t code it.

Managers, HR departments and headhuters need to stop taking shortcuts in hiring because it only leads to hiring mistakes.

Team interviews that make a candidate discuss technical topics with people who can call bullshit on them is the best way to find the best developers for the team. Unfortunately, it is also the most expensive.

=C=

Rob... (@akrabat) wrote at 11/2/2009 4:41 am:

A lot of people can “talk the talk” and it can make the interview very long if you are trying to work out if the candidate really can do it rather than being able to spout the theory.

Having been burned once, we now require coding tests during the interview that are fairly simple if you really can build a website in PHP without copy/pasting code from hotscripts. We still have over 50% failure rates though.

The biggest advantage of a ZCE to me is that it shows the candidate takes their career in PHP seriously and wants to be more than a Drupal configurator. They still have to take my test though!

Regards,

Rob…

Cal Evans (@calevans) wrote at 11/2/2009 6:31 am:

@Rob

I agree and disagree.

I agree about ZCE. if nothing else it shows that they put forth the effort. I question the value when talking about actual knowledge but it does show that they take it seriously.

I disagree about the “talk the talk”. I’ve never hired a developer who went through a team interview with at least 5 people, who could withstand the scrutiny and not be the real thing. The secret is that I’m not relying on my personal knowledge or gut feeling. Nor am I relying on individual interviews conducted one-on-one. The dynamics of a team interview are such that it is very difficult to BS other developers for very long.

That’s not to say that I’ve never made a hiring mistake. Lord knows I’ve done that enough. Most of them, however, can be directly attributed to me not following my own rules.

IMHO, etc.
=C=

Rob... (@akrabat) wrote at 11/2/2009 6:57 am:

Cal,

You’ve always worked for bigger companies than I have :)

I can’t afford to take my one PHP developer out to interview someone unless we know they are likely to be good. We always do a “team” interview, but only after they have proved that they know how to code.

Regards,

Rob…

Brandon Savage (@brandonsavage) wrote at 11/2/2009 7:25 am:

Rob, it’s been my experience that there’s a fine line between a coding test that’s over the top, and a coding test that’s acceptable.

Asking someone to write a recursive function, for example, is an acceptable technique to weed out programmers from non-programmers.

Asking someone to build you an application that stores information in a database, emails it, validates it, etc. and requires the use of half a dozen PEAR packages, is over the top, however. I’ve actually experienced this recently. To do the kind of job I’d want to do on that kind of test, it would have taken me three or four hours to complete (I’m not going to take OO PEAR packages and write a procedural app, for example).

Contrary to what some people seem to believe, I have no trouble submitting proof that I’m capable of developing, and I’ve gotten up during interviews and headed to the white board to describe something, or to write a function. I do have a problem spending several hours working on something, though, partly because I’ve already spent hours developing this blog and other code samples, and partly because I don’t have hours and hours to give.

I like Spolsky’s model best, because I think it does hit the major points. He says “you don’t want to give them any problems that take more than about 5 lines of code; you won’t have time for that.” I think he’s right. You can weed out the bad candidates without making the good ones sit in your office for hours working on a project just for the honor of talking with you about the slim possibility of joining your team.

Developer Art wrote at 11/2/2009 7:29 am:

I rather see these coding tests as childish. If you have any doubts a person can write code why invite them in the first place?

Coding tests can be perceived offensive and disrespectful by decent programmers. Especially by those who come to take a mid-senior position. They expect some serious talk about software design, process and quality control, but not to be asked to write a simple loop to prove they could code.

If you wish to learn their coding style, ask them for sample code. A better way would be to give them a try and see how they run. That’s what the evaluation period is for.

Cal Evans (@calevans) wrote at 11/2/2009 7:39 am:

Rob,

I agree that pulling all your developers into a meeting for an hour is an expensive proposition. However, making a mistaking on hiring is even more expensive. Especially for the person you hired. I’ve always felt it’s worth the code, small company or large.

Dart,

Speaking of evaluation periods, that i the other part of my strategy I always fail to mention. Before I will offer someone a FTE gig, I offer them a 30 consulting job. I always pay well for the consulting gig but about 3 weeks in, I get the entire team together again for a stand-up. We have a quick discussion of the person, how they are performing and most importantly, how they are fitting into the team. At that point, we the team (not me the manager) make the decision to hire or not.

=C=

p.s. for those that don’t know us, Rob and I are god friends. That’s why I feel comfortable having this discussion in public with him. Don’t get the idea that I don’t respect Rob. he’s on my short list of people I call for advice. :)

Rob... (@akrabat) wrote at 11/2/2009 7:48 am:

Brandon,

I agree completely,

Our test has a couple of code inspection questions and a three “write a function that does X” ones. I’d probably skip it for someone who has a proven track record with an open source project where I could inspect their code.

Cal,

Yeah! That’s why we always “team interview” the ones that are “likely to get hired” :) I just can’t afford for my “team” to be saying “why did you waste our time on that joker?”. I want them to be exploring the limits of the candidate’s abilities and ensuring that their personality will fit us.

With our new designer, we did the “contract for us first” approach and that worked really well. I’ll probably try that with our next dev hire too.

Regards,

Rob…
(P.S. Yep – and it’s a useful addendum to Brandon’s post as we’re exploring the pain of hiring from both sites!)

Eli White (@EliW) wrote at 11/2/2009 7:59 am:

Chiming in here:

To Brandon: Great article, I’m there with you, as you know.

To Sudheer: I disagree. For example, you state that you had that person spending an hour. And only then once he’d failed asked: “Where does a server store mail on the server”, and got the ‘mutt’ answer. Had you asked that question in the first place, you could have sorted that candidate out much quicker. Or even had you asked him more theoretical questions, “How would you configure a mail server” – just having him walk through the paces, perhaps even with prompting from you, mentally/verbally.
That’s one of the core issues I have with coding tests, as such. It seems always more efficient, and IMO more revealing, to interview them about their knowledge of code instead.
I see the ‘coding test’ as a crutch. If you were interviewing a ‘manager’, you couldn’t give a test, and would have to rely on your interviewing skills.

To Daniel: Agreed, and that’s why to me, ‘coding test’ and ‘provide sample’ are very different things. Most everyone of mid+ level can probably provide a code sample. Even if it’s not a ‘good’ sample. Just something that they had written once. And that the interviewer can then look over, and come up with lots of questions based upon it: “Why did you do this?” “Interesting technique there, was there a reason for it?” etc.
Without putting them on the spot, and requiring a code test (whether in person, or offsite)

To Bron: Those of us who have ‘resume slung’, Thank you for not ;) The reality of the situation is, you can’t tell/research everything that you possibly want to know about a company, until you’ve sent in that resume (or made contact in whatever way).
You don’t know if the company dynamic will work for you, you don’t know if ‘little things that matter’ are going to be leaning the right way. Even more importantly you don’t know if the company is going to like you, willing to pay enough, etc.
So yes, some resume slinging is at times needed. Now, I’m not talking about situations where you are semi-content at your current job, and are just trying to look for greener pastures. But when you are in the ‘I need a new job – stat’ situation, as Brandon is in.

To Cal: *high five* I agree completely that team interviews are crucial. For some reason a number of the last places I’ve worked instead of wanting to do a team interview, didn’t want to ‘stress’ the person out, and so instead only had lots and LOTS of individual 1-on-1 interviews instead.
I highly dislike this practice. While a couple 1-on-1 might need to happen (with the boss, etc). Having the primary interview be a team one, means that the entire team can play off each other, making sure, as you said, that they aren’t snowed over. But more importantly everyone on the team sees the same interview.
There is nothing worse, than doing 5 1-on-1 interviews, then getting together afterwards, and having 4 people saying “OMG, great candidate!”, and one saying: “WTF, that person was horrible, didn’t know a thing I asked them”. The 1 assumes the 4 were snowed. The 4 assume the 1 is an idiot or nitpicky. In the end, you don’t have a solid ‘overall team feeling’ about the candidate.

and finally:

To Rob: I understand about not wanting to pull your developer out. However, it seems that much of that screening job, falls upon the manager. To have had an initial phone interview with the person, to get a ‘good feel’.
However I’ll also say that it sounds like you are interviewing much lower-grade people (entry level), if you are seeing 50% failure rates on your tests anyway. And I DO agree with giving a coding test (though preferrably still a ‘take home’ version), to entry/junior level programmers. Just because of that fact, you will get alot of applicants who simply aren’t qualified in the slightest.

*shew*

Kyle Cordes (@kylecordes) wrote at 11/2/2009 8:35 am:

Over here, we use a variation of the “coding test” approach: after choosing some applicant using short initial interviews, we *pay* the applicants to write some code, typically 1 day of work or less. I agree with the sentiment that it is inappropriate to ask applicants to do significant work unpaid.

Applicants send us their code, and a short screen-video tour of their code, typically ~5 minutes or less. It is *amazing* how much we can learn from watching that 5 minute video. “hire” or “no hire” is often obvious in the first minute.

Sudheer (@bngsudheer) wrote at 11/2/2009 8:39 am:

@Eli White: “Had you asked that question in the first place, you could have sorted that candidate out much quicker.”

The point I’m to trying to make is, Red Hat certified engineers can talk jargon. You certainly can make out whether the candidate belongs to A grade category, by talking about relevant subjects for a while. Every hiring manager wants to pick the best of the best candidates available in the market. In my experience, it rarely happens. You’d want to consider offering the job to a slightly less skilled programmer/system administrator. But not the kind of people who can’t write rudimentary code.

In the incident I mentioned, the candidate correctly answered most of the questions I asked prior to the test. There were also some incorrect answers. I couldn’t exactly determine the candidate’s skill level based on the discussion. The test revealed the fact.

Had I known the fact that the candidate did not know where the emails would be stored on the server, I would ask him that question before giving the test. The person managed to successfully answer some of my questions because he had worked with a team that managed several email servers. Unfortunately, the candidate didn’t have a clue of how the servers were setup although he performed many operations on the server. I now understand those operations were nothing but a series of instructions provided by a senior.

I know a lot of people who have passed RHCE, MCSE and CISCO certifications but don’t know the basics of those technologies. I even asked a person, how they managed to pass the test in the first place. “Dumps” was the answer. A set of question papers are available in the market. If you manage to by heart answers you’ll sure get the certificate. After obtaining the certificate if you manage to land a job as a junior, you will be more familiar with the industry jargon and theory. Some people abuse this knowledge and claim higher skill levels. The test clears up the mist.

Elizabeth M Smith wrote at 11/2/2009 8:45 am:

Amusingly enough I just had this conversation with some people I know…

I think a coding test can be appropriate in an interview situation. But my definition of “coding test” does not include anything that should take over 30 minutes.

In fact, I think the BEST coding test should have a time limit – and test not only what the candidate knows and how they think – but how they LEARN.

As Brandon mentioned the best programmers never stop learning, never stop growing. Teaching them something new on the spot – “here is the class we use to access the data store for our system – this is the general way we use it – write a short script utilizing the class to store/retrieve and validate data” is far more useful then wanting a “mini-application”. Give them 20 or 30 minutes, and then do a code review on the spot to see how they handle criticism!

My 2 cents on the matter ;)

Duane Gran (@duanegran) wrote at 11/2/2009 9:13 am:

I agree with Cal Evans about the team discussion as the primary weeder, but there are many organizations that hire one or two developers. They don’t have a team per se and they are looking for some form of signal — likely because they have mis-judged a previous hire.

The up front tests effectively weed out the bottom quartile but as Brandon notes, it may be at the cost of also weeding out the top quartile. These tests indicate a culture that plays it safe and lacks the ability|desire|competence to discern the A Players in their application pool. Possibly the hiring organization has deemed this a reasonable trade off, but I agree with others that they would better off doing something more creative.

Keith Casey (@CaseySoftware) wrote at 11/2/2009 9:52 am:

Interesting that no one points out the obvious part…

I’ve worked/talked with a number of companies where the people (even the team leader) were clueless technically. I worked on one particular team where one of the leads didn’t have a clue on OOP or basic database design.

What do you do then?

Marcus Ward wrote at 11/2/2009 9:57 am:

While we’re griping about the inordinate amount of time required by coding tests, why not lump in the absurdly complex resume submission requirements some companies have. Do I really need to register at the HR dept of every place I put a resume into? The last one I applied to wanted 15 years of detailed work history, it took me 2 hours to fill out. It seems as though these systems are designed to weed out the intelligent people who are offended by the waste of their time and keep only the sycophants who happily jump through all the hoops.

Cal Evans (@calevans) wrote at 11/2/2009 10:01 am:

Keith,

That’s easy, you contract with me to do your tech screening. :)

Seriously though, if you have no clue about the technologies you are hiring for, your company has bigger problems than coding tests.

“If you have never been a developer, you have no business managing developers.”
– Cal Evans

Brian (@None) wrote at 11/2/2009 10:54 am:

These coding tests should be done on a machine, not a pen and paper, with net access, and be testing how they solve a problem.

The test that irritates me the most is a list of questions on paper: what does the PHP function X do, what PHP function would I use to do Y? Only a moron memorizes php.net, and only a bigger moron thinks by doing so makes you a good programmer; being able to answer these types of questions demonstrates no programming skills. These questions only show how little the interviewing company know.

Would you insult a journalist by giving them a test to define words from the dictionary…

Carl Seleborg wrote at 11/2/2009 11:18 am:

Brandon,

Those are interesting points you make, but I disagree, at least within the context of software product companies. Let me explain:

Your plumber analogy is simply wrong. Hopefully, the developer you are trying to recruit will have more impact on your company than the plumber will have on your home. Asking the plumber to build a grid to prove his skill is certainly overkill, but asserting a candidate’s skill can, in the case of small companies, make the difference between succeeding or going out of business. I know, I’ve seen it happen.

If you are looking for a place to work, you should welcome those tests, not throw them away. Frankly, at the time I was looking for a job, I went to a dozen interviews where skill assessment took to the form of an HR person looking me straight in the eyes and asking me if I was good. I no longer want to be a software developer at such companies.

On the other hand, my current employer, Ableton, is a hugely successful music software vendor, with a reputation for quality. Guess what? We ask candidates to send in a programming exercise. Not a complete application, mind you. The candidate has to write two functions related to our audio time-stretching feature. They have to look at the demo, see how the feature works, and re-implement the computation. In fact, the recruiting process is quite demanding on the part of the candidate. I had to “gamble” 4 weeks of vacation out of the 5 I had every year to do an internship there. I made it and am now working with 100% competent colleagues, with which I learn new stuff every day.

So my guess is: as much as it is true that those programming tests require you to spend some time on them, that time may very well be your best investment while looking for a job you’ll really like.

Cheers, and keep up the good writing though.

Jack9 wrote at 11/2/2009 11:39 am:

> I don’t know about Zend PHP certification yet.

I do. ZCE is anithetical to demonstrating programming knowledge. I would venture to say, it’s horrible for demonstrating PHP knowledge, unless you value knowing the outcomes of unusable tricks and horrible coding practices as “knowledge”.

Anonyous wrote at 11/2/2009 11:57 am:

“It’s disingenuous to the value of their time, and disrespectful, especially to mid- and senior-level candidates, because it assumes that they don’t have an understanding of the language and they must prove to you otherwise. A coding sample, in the case of a mid-level developer, or a quick few phone calls, in the case of a senior developer, should establish pretty quickly as to whether or not they’re qualified.”

I would really, really like to know about these phone calls that can establish the credentials of a senior level developer.

When people call themselves a “senior level” developer on their resume, we really have no idea what that means. Is his idea of “senior” the same as ours? There’s no uniform standard or certification for that. The only way we can tell is from their actual work.

The plumber example is flaw for the reason that hiring a developer is establishing a long term relationship with that person whereas a plumber will work for a relatively short period of time. The cost of hiring a bad developer is much higher than the already expensive salary. A bad developer will contribute nothing to your project at best while taking up a valuable head count. Worse yet, he can derail it.

While the format and length of tests are debatable, I have yet to find a better way of discerning a good developer from a bad one than to look at their actual work. No amount of “soft” questions or theoretical questions has been able to replace it. Asking the specifics of some method in some library is extreme and probably useless since that kind of information can be Googled. Seeing how they approach a problem and how they solve it, however, is very insightful about that person.

Joe Stevenson wrote at 11/2/2009 12:23 pm:

As a hiring manager that has asked senior developers the question “What is the difference between recursion and iteration?” and received both blank stares and horribly wrong answers — I think seeing code they have written is crucial. Unfortunately programmers don’t tend to make portfolios, and I have seen more than a few slick talkers sneak their way into developer jobs that they are not able to perform — something a simple test would have weeded out, assuming they don’t offer sample code to start with.

Plumbers are a bad examples, if I hire one that is unsuccessful at snaking a pipe I will not pay him.

Keith Casey (@CaseySoftware) wrote at 11/2/2009 12:59 pm:

@Joe & @Anonymous

I think that’s where Open Source developers will always have a better advantage. They can point and previously developed and publicly available code samples. In fact, with a bit of poking, you can find out *lots* more than even a few interviews can give you.

You can see if they fix or add subtle bugs. You can see their interactions with others – both within the team and outside of it. You can see some of how they’re involved in the project. Do they jump to solve problems or just wait and watch? Do they encourage others and help them improve contributions? Do they work to mentor and direct new people?

Those are all attributes you want in a senior developer.

Anonymous wrote at 11/2/2009 2:01 pm:

Totally agree. I’m a consultant and therefore I move from company to company a lot based on projects. Only recently have I started seeing companies use “code tests”, which I find a waste of my time and an insult as I’m being hired as a Lead Senior Developer and Architect. Both tests I had were for gaming companies, one asked me if I could develop space invaders, but not in details, just say it in 5-10 minutes. I did that and they offered me the position on the spot, but they could have just asked for code samples.

The second interview at another gaming company is the one that ticked me off. The person conducting the code test on me was another developer in the company, and he was there alone, no director with him or anything. First he wanted me to write the code on a freaking whiteboard, my penmanship is crappy as it is, and he went as far as making sure I wrote down return types on functions (on the freakin whiteboard).

After I finished one snippet, he asked for more, eventually he asked me to write design pattern code (singletons, mvcs). I was like “dude, are we going to be doing this all day? I can tell you how to write these classes, I don’t have to write them on a freakin whiteboard”. Then he asked me a question about a random class in the programming package, (one class out of thousands), which happened to be one that I would never use. I simply responsed “Are you serious? I would just look it up in the language reference manual. Do you think I memorize every single class out of the thousands that are available?”

Eventually I got tired of it, and walked out on the interview. I think he may have done it because he felt his position was threatened, the company should have had a director with him.

Would have saved me so much time if he simply asked for a code sample. Seriously, what a waste of time.

Chris Wenham wrote at 11/2/2009 4:48 pm:

I use code tests in all the interviews I give, even for senior level developers, because it weeds out those who genuinely cannot program.

I have personally interviewed “Gurus” with 20 years experience who didn’t even know that you had to create an instance of a class before you could invoke its methods. It was after letting that one go and forcing a re-write of everything he’d done that we chose to perform code tests–no matter how “senior” their resume said they were or how many code samples they had in their briefcase.

Your comparison to electricians building a small electrical grid doesn’t fit: electricians are hired for contract jobs that can be over in a few hours or days. Since we’re planning to hire you for at least 5 years at a $65k floor or higher, then invest substantially more in training, we _WILL_ want to see for ourselves that you can do the job.

If an electrician screws up, we know he’s insured and we can sue for damages. If an employee screws up because of incompetence, we can’t expect to recover damages by suing him.

Furthermore, if we were planning to enter into a 5 year contract with an electrical firm, one that would be worth millions of dollars over the full term, you’re damn right we’d have them demonstrate more than just wiring up a light switch.

That’s why the tests are different.

It’s not sufficient to just review code that you claim is yours, we need to see you write it to remove all doubt.

Your complaint that coding tests aren’t enough to judge a programmer is a straw man. Code tests aren’t the interview. We DO ask more than one question and serve them more than one kind of test. Of course a code test won’t tell us how well you learn, but that’s why an interview is an interview and not an exam.

And finally, if you’re tired of doing 15 code tests in search of a job, then maybe you could try McDonalds. I wouldn’t be interested in hiring someone who just wants to spam job applications. We want applicants to choose us as their partner for their career, or at least the next five years of it. We’re going to be dedicated to them, and we aren’t going to hire unless we believe they’re going to be dedicated to us.

Daniel Francis wrote at 11/2/2009 6:33 pm:

If you’re a programmer and can’t tell if the person sitting opposite is a competent programmer within five minutes, you aren’t qualified to write code yourself.

Sean wrote at 11/2/2009 8:33 pm:

99% of the programmers we’ve fired we’re experts who could easliy pass any coding test we could give them. However, all of them were fired for reasons unrelated to their technical skills. We need a good test for lazy, lying, sleazy, rude programmers who piss off the whole team. Those are the people we need to worry about. It’s much easier to improve an employee’s technical skills (especially when they are eager and motivated) than it is to improve an employee’s poor work ethic.

Alan wrote at 11/2/2009 10:01 pm:

Have hired a lot of people, and i can say 99% have worked out great. During the interview I use the following :

a) put the resume down.
b) For senior – ask them – explain to me (using white board etc) on a few large projects (has to be more than one) what mistakes you made, or what you would change. Here you are testing a) their ability to communicate – what is the thing they built, why is what they are proposing a mistake or an area of pain (smell in jargon), b) technical ability and c) ability to learn and d) the depth of their DEVELOPMENT experience (not language experience). Get them to outline the design, how it solves the pain etc. Ask questions (but not leading questions). You are looking for the design process, requirements, expectation management, how team members were used etc, how they communicated up and down. Why tech decisions were made. And if they can’t explain more than (oh, it works) on a tech decision, odds are a) they didn’t make it b) or don’t understand it and c) can’t communicate it. Listen in particular say how the tech relates to the development process, the problem at hand etc. You will also here if they take responsibility, or blame others. Listen for these clues.

And don’t ask leading questions.

If you cant’ do this , you should be interviewing senior people.

Why do ask about the mistakes/areas you would change ? In life, we learn from our mistakes. If you haven’t done it, you generally can’t talk about you mistakes, which means you probably haven’t done it.

(note – we do the same for their successes).

PS. we also give a basic coding test – ie. write a few lines of code to show they can code.

Richard Lynch (@LynchRichard) wrote at 11/2/2009 10:53 pm:

I must also add that for a really good candiate, they’re going to get an offer long before they finish your silly pre-screening application “test”…

At least once, I sent in a half-finished “test” with a note that I had another offer, so here’s what I have done so far.

Never heard back from them one way or another, which after a phone interview and several hours on their “test” seemed pretty rude to me…

I must confess I also have almost nothing in the way of current code samples — It’s all proprietary work, and I couldn’t possibly consider sending it off as a “sample” when it’s not legal to do so!

If you can’t figure out whether you want to hire me just from Googling my name and finding my work online, then you probably don’t want to hire me.

michael kimsal (@jsmag) wrote at 11/2/2009 11:05 pm:

I’ve had some odd experiences in being interviewed. I’ve only been given two ‘take at home’ coding tests, and both were (imo) ridiculously over the top in complexity – each indicated they expected the test would take 12-15 hours. No thanks. I spent a few hours on one, sent it in, and didn’t even get the courtesy of a reply.

Most other tests I’ve been given ask for writing *real* code on a whiteboard. No keyboard/computer involved, at which point I invariably point out that I’m not very good at whiteboarding real production-ready code, and that I’ll be writing pseudo code.

One company I interviewed at (and was actually hired by) actively refused to look at my code samples, even after I offered them to multiple people during the interview process. It “wasn’t fair” on the other candidates, because not everyone had code samples available to bring to an interview. How’s that for reasoning?

I’ve also noticed the same initial screening questions (which I inevitably never prepare for and get wrong) involving differences between inner and outer joins and other somewhat geeky stuff. I’m perhaps too much of a generalist consultant to have all this stuff ‘top of mind’ day to day and tend to flub these questions, although I can send over code samples of various joins in action in real projects.

The team interviews are generally a good filter, as Cal suggests, because you can’t easily BS an entire group of people. Additionally, you’ll all have a better idea as to whether there’s a personality fit or not. Whether the tech chops are there or can be learned may be secondary to having to work day in and day out with people you don’t get along with.

michael kimsal (@jsmag) wrote at 11/2/2009 11:11 pm:

@jack9

The ZCE is really a mixed bag, and means different things to different people, especially employers. The biggest think you can take away from it is that someone has some element of ‘seriousness’ with which they regard their career to take and pass the exam. Not to say people who don’t take it aren’t serious, but it’s one measure people can use.

As to the things it tests, I’ve now come to look at it as a way of demonstrating that you’re able to deal with a myriad of other peoples’ code, which is often what employers are hiring you to do (come in and work on existing code). There’s so many weird gotchas in PHP, and various bad ways of doing things, that *not* addressing some of those in the ZCE would be ignoring a practical component of day to day work which most people have to deal with, if only in ‘refactoring’ or ‘cleanup’ mode.

Jon wrote at 11/2/2009 11:40 pm:

I think you’re being not only a baby here, but also a bit pompus.

I’ve read your articles. You aren’t god.

Plus, if your unemployeed, what the hell do you think you’re doing for your future “potential” employers by sounding like a jack ass? You are now gonna tell THEM how to do their interview process?

Jesus brandon, give me a break. Companies do these tests to know what you’re capiable of. And, quite honestly, if you were to tell me this after I gave you a “write some test code”, I’d tell you “there’s the door. have fun. and don’t bother calling us, we’ll call you”.

The reason is because while it’s good to be confident of your work, being self righteous doesn’t do anything but leave a bad taste in people’s mouths. Especially when it makes you sound like you’re better than everyone else.

Rafael Dohms (@rdohms) wrote at 11/3/2009 5:57 am:

I have been on both sides of the coin and on several different techniques.

I totally agree that the coding tests are awful especially if its the “write a whole application”. I have taken this test once, and even with my years of coding i struggled getting a badly configured machine to run the code which i was forced to write on notepad! That’s really disrespectful, i was hired, but i soon after quit as the test sort of reflected the rest of the job.

I have also taken the questions test, less stressful and maybe more to the point, seeing as the 2 well known developers (me and a friend) finished in 20 minutes and everyone else took 1h30.

But still, especially on a senior level, coding tests are not even worth it, the best way to select a senior developer as i see it is based on his activity. If he contributes to OS Projects you can surely get a feel of his skills, is he has a ZCE you can get the feel for his focus on the career/language, combine those with his role in the local/intl. community and you can read him completely.

I also agree with Cal on the interviews, our current interview process is to give the guy at least 2 or 3 1-1 with the whole team, which is awesome since we are a multi-discipline team, each one can evaluate a different area, plus we rely a lot on the “team feel”, the guy needs to be good, but he also needs to be a great guy on a personal level, like fit in the team.

This has worked great for us, we have removed all the weeds from amongst us, and the only 2 people which were not approved based on this system, well they did not last long.

Matthew Weier O'Phinney (@mwop) wrote at 11/3/2009 6:39 am:

I personally would never ask for a code sample during an interview or when asking for a resume. That said… I *would* likely ask for links to some open source code they’ve written: project links, github account, etc. This provides me with an invaluable tool for evaluating their skill: I can see both the quality of code, as well as how they interact with others. The two combined give me a much better idea of the abilities of a developer than any code test could likely provide.

Elizabeth M Smith wrote at 11/3/2009 7:51 am:

Open source DOES make a difference
And I think it should change subtly the way interviews work
Because if an interviewee has open source code and experience, their participation should factor into how the interview is done!

Having a portfolio of work to show IS something a developer needs – I also have walked away from interviews that required “coding tests” because I have a lots of places online I can point to and say “here’s a bunch of my code… in several different languages” and no I’m not going to do something that takes me more than an hour to complete when I’d rather be hacking at a new PHP extension ;)

Eli White (@EliW) wrote at 11/3/2009 8:26 am:

To Michael Kimsal:

Wow, yeah, a 12-15 hour test is ridiculous. Back at Digg we used a standard ‘take home’ coding test when I was there, for junior->mid folks. It was something so simple, that it could/should have been done by a competent programmer in 30 minutes.

That is, if they just did the basics, and didn’t overengineer it. Which to me, was part of that test. Are they the type to completely overengineer a simple task? Or can they just crank out some basic fugly code, but that does the task asked. It, to me, is a cool/basic test just to see how they ‘think’ when they code.

But, we didn’t ask this of ‘senior’ (unless we got suspicious in interviews), and if someone had OS contributions, or just wanted to provide a general code sample, we’d be fine w/ that as well.

Eli White (@EliW) wrote at 11/3/2009 8:32 am:

To Elizabeth:

I too have walked away from an interview when asked to do basic coding examples ‘on the spot’. Not answer questions about my knowledge, but “While I look over your shoulder, write a recursive function that detects a palindrome” kinda stuff, to “See if I understand recursion”

Realize, when I’ve gotten those questions as a junior, even mid-level, I had no problems answering them. But when I was walking in w/ almost 15 years experience, and interviewing for ‘Big Picture’/Architect/’Direction Setting’ type positions, it really had no bearing (or shouldn’t have) on how well I was going to perform my tasks.

I saw it as an indicator that the person interviewing me either didn’t understand the skillset I was bringing to the job, or that the job wasn’t what I ‘thought I was interviewing for’, and was going to be a more junior-level code-jockey position in the first place.

So I ended the interview.

(And anyway, doing that solution via recursion is inefficient. A simple loop would be better 90% of the time. Or better yet, go for simplicity & maintenance, and just do: if ($var == strrev($var)

Michael Kimsal (@jsmag) wrote at 11/3/2009 8:36 am:

@eliw, I completely agreed – 12 hours on a coding exercise was crazy. Sure, they wanted to ‘weed out’ the low-end people, but they’re weeding out qualified people who simply don’t have that sort of time to devote to something. Those sorts of tests have a habit of reinforcing the company’s culture by ensuring only like-minded and like-abled people get in. While that’s not necessarily a *bad* thing, it wasn’t helpful to me at the time :)

Wow, that’s almost a trick situation, where you’re expecting someone to submit ugly code that was just ‘cranked out’. When I’ve done that in the past I tend to get raked over the coals by someone with less experience who likes to point out all the ‘mistakes’ I made on the whiteboard (to which I’d like to ask how much *real* coding they do without their editor, a keyboard, a screen, or for that matter unit tests).

I may try that if I’m ever in a situation again where I need to whiteboard. I’ll just whiteboard out unit tests first, see if they even get what I’m doing.

Anyone else ever done that? For all the talk of ‘agile’ and such, people still expect you to write ‘working’ code, on a whiteboard, *without* unit tests! I have a hard enough time writing working code on a computer *with* unit tests :)

Michael Kimsal (@jsmag) wrote at 11/3/2009 8:44 am:

@eliw again –

I’ve had those ‘write a function to reverse a string’ questions, and sometimes I answer them and sometimes I don’t. My answer is something like “the times when I’ve tried to roll my own solution for something that’s already out there has usually ended in a huge mess – it’s better for all involved to use documented, well understood packages/libraries/functions/etc.” But that’s me thinking holistically, and honestly from a managerial and business owner perspective, not a code monkey perspective.

At the same time, employers do need some way of knowing that you understand how to implement basic, well understood ideas in algorithms. I guess published code, open source stuff, previous portfolio work should do the explaining for you.

The other question I’ve had repeatedly is “how would you parse through a file and pick out all the unique elements from the third column in the file then sort them. From a command line shell”. I know they’re looking for some “I’ve done sysadmin work so I’m some unix geek and I know sed/grep/sort/uniq like the back of my hand and can pipe stuff around in one long BASH line” sort of answer. I generally acknowledge that that *can* be done in a variety of shells, but I tend to do complex shell scripting in PHP (or on occasions ruby, groovy or perl). There’s a wider range of things I can do in PHP/ruby/groovy/perl (connect to DB, easier logging, etc), it’s one less language to burden other team members with, and so on. Trying to impress someone by writing everything in one bash line is just crazy. For some reason people praise ‘well documented, well written OO PHP code’, but the same people (sometimes) want unintelligble one-line BASH hacks. :/

tinycoder wrote at 11/3/2009 10:16 am:

I think someone should step forward (Google / Canonical / RedHat / MindTouch / Amazon / Yahoo most likely) and hire guys who have contributed to opensource projects, visibly and therefore can demonstrably and arguably write good code as well as maintain it.

This sets a great precedent and we can only hope that this has a trickle-down effect. I know it sounds cliched, but this is a long-term thing, just like open source adoption has been long-term.
The make-or-break nature of exams is clearly a reflection of abundant supply of programmers versus very limited demand from software development companies.
To fix this problem, unless the change comes from top downwards, no significant change will result.
Hence, the companies I named.
To start a web dev career today, all I need is a good internet connection. It’s come to that stage. Now if only this ease could be tied up to the hiring process, it would be a worker’s economy, not the feudal system it is today.
And your old trusty Craigslist will help fill the gap in between those two segments

sean wrote at 11/3/2009 10:35 am:

I work at a small company. We have been burnt before by applicants saying that they are “Architects” or “Senior”. Applicants use these terms too loosely. Just because you have been code for 10 years does not mean you are an architect or senior developer.

We break our interview up into 2 parts: A technical group interview and a technical group coding exercise.

During the technical group interview we ask the basic questions (difference between inner and outer join, how to do string manipulations, how to setup a web site). Then we ask about design concepts that the applicant says they have understanding of to start a debate ( where is the best place for page validation.. server or client, test driven development, Pros / cons of Agile, waterfall or extreme programming, NHibernate, LinqSQL etc )

If they pass the technical interview, then we do part 2…

During the technical group coding exercise we have a member of our team be the code writer as the applicant is the designer. As the applicant designs how to solve the problem, our team member codes, our team member will ask questions like “will this be the most efficient?”, “have you thought about the problem this way?” This exercise helps our team understand how the applicant will interact with us. Also many times it shows us how the applicant will react when there is a problem they cannot solve. Will they go to Google, a blog site, or will they want our team member to solve the problem? Does the applicant start a good debate on how he designed the solution?

Yes this interview process take a minimum of 4 hours, but it weeds out a lot of the jokers.

Mark Kimsal wrote at 11/3/2009 10:47 am:

I generally don’t like coding tests because the people administering them don’t know how to interpret the results. Usually, I end up asking them to clarify the test to the point where nobody can give an answer and they say, “oh just do anything”.

It’s a huge time waster when you *know* that they’re looking for 1 particular answer and you know that you don’t code that way. It also makes the company look bad. People say things like “write a function to reverse an array without using php’s array_reverse()”. WTF, I automatically don’t want to work at the company. The level of incompetence is so high that they can’t even construct a basic coding challenge without looking like a bunch of idiots.

What do you do when your level of coding is so far above every other person’s already employed at the company?

Mark Kimsal wrote at 11/3/2009 10:49 am:

I also completely disagree with the ZCE comments. ZCE is about conforming to Zend’s style of coding and strange views on “optimization” which don’t pan out in the real world. Anybody with a ZCE is usually on the bottom of my list because it means that they don’t really code in other languages and PHP is their one track of development, and Zend is their one methodology, and everything Zend does is awesome by default because “they made PHP” or some other crazy nonsense.

Daniel H (@dhirschl) wrote at 11/3/2009 11:14 am:

I’ve applied to a job before where I had been asked to answer about 10 conceptual and function questions (such as structure of cursors in SQL) before I’ve been asked to come in for interview.

After answering the questions, I was asked to come in to take a coding test that consisted of: creating a database, stored procedures, designing web services and a web front-end.

Following this test, I was asked to come in several more times to discuss the psychological questions: what are you strengths, etc. and also to talk about my experience.

In the end I didn’t take the job because I didn’t care for the company culture.

Anon wrote at 11/3/2009 12:09 pm:

Some good comments here. For those of you posting here who do the interviewing, how much does a college degree factor in to how much you decide to probe a candidate? I can see the value in coding tests for developers who frequent PHP blogs as the world is filled with “web developers” who are self-taught people who bought a PHP book on amazon without any real schooling on computer science.

Also I have to say I agree Jon above. If someone were interviewing to make a career move for the better I can see them being picky and having some leverage throughout the interview process. However, reading the blog of someone who seems to come across some unfortunate circumstances, I would have expected to see a bit of humility and some willingness to jump through a few hoops potential employers present in order to get back on his feet.

Matthew Weier O'Phinney (@mwop) wrote at 11/3/2009 2:09 pm:

@mkimsal The ZCE doesn’t talk about coding standards at all. It talks about language features. Yes, some of the information is out-dated and/or inaccurate — but to say outright that you’d put any ZCE at the bottom of the pile is, I feel, doing yourself and your applicants a disservice. You need to look at the entirety of their resume, not just one aspect of it. Are you going to refuse to hire the likes of Marcus Boerger, Ilia Alshenetsky, Tobias Schlitt, etc — because they’re ZCE’s and/or helped write the exam? Get real.

The reason the ZCE exists is because, particularly at the time it was introduced, there were many junior PHP developers, but not much to differentiate one from another in terms of basic knowledge — and one way this has been done for other languages and disciplines is to provide certifications. If you disagree with the methods of the certification, then simply disregard it when looking through candidates — but outright dismissing a candidate for taking it? That’s doing nobody any favors, least of whom your employer.

Michael Kimsal (@jsmag) wrote at 11/3/2009 2:22 pm:

@mark – the ‘what do you do when you’re more advanced than others at the company?’ – was addressed a bit earlier, but basically, probably don’t work there.

re:zce – I think the zend *framework* cert is probably more like what you’re thinking. The standard ZCE isn’t about ‘coding standards’ specifically.

Ben Dunlap (@bdunlap) wrote at 11/3/2009 4:46 pm:

@mark said ‘People say things like “write a function to reverse an array without using php’s array_reverse()”. WTF, I automatically don’t want to work at the company.’

Joel Spolsky argues for this kind of thing as a quick & cheap way of verifying that nuts-and-bolts programming is second-nature for a candidate. Any qualified programmer, he says, should be able to produce that function about as quickly as it takes to write it down. Someone who has to think about it at all is probably not the best candidate.

I suppose an ideal example wouldn’t replicate a built-in function, but in general the concept seems reasonable: ask for something stupidly simple just to make sure the person really “has” the language. It’s a quick filter, nothing more.

James wrote at 11/3/2009 5:06 pm:

I, too, sit the “other side of the table”. I, nor my Technical Director, would dream of hiring someone without asking them to fill out a written test.

You can ask them questions about code and technology all day. But we don’t discuss code all day in the real world, instead they need to understand the business & technological problems presented to them, design and implement the solution. Our test is not solely focused on PHP either, we ask networking and database questions too.

We would never ask for a complete application or even class/script. The most they need to do for any given question is about five lines of code. Usually a few possible answers but the question will be laced with hints as to problem they’re trying to solve. We even allow access to the PHP manual, after all they’ll have it under normal conditions.

To answer anon’s question about degrees, they are to be treated with caution. One guy had a better BSc grade than I did yet had no concept of a mutex. Point is you probe at what you expect them to know rather than make an assumption.

anonymous wrote at 11/3/2009 6:15 pm:

@James said “To answer anon’s question about degrees, they are to be treated with caution.”

Ain’t that the truth. I’ll never forget asking a credentialed co-worker once what he knew about the STL — I was looking for some quick guidance — and getting the reply, “Nothing, my professor wasn’t really into C++”. He had been out of school for a few years at that point.

It wasn’t the “nothing” that flabbergasted me, but the reason given. “Did you stop learning when you got your degree?” I didn’t say.

Ivo (@Ijansch) wrote at 11/18/2009 9:30 am:

At my company, we’ve done recruitment without any coding test for about 9 years and have introduced coding tests a little over a year ago. Since we introduced them, we were able to better screen our candidates and make our recruitment process much more efficient.

Our test is designed to be simple; it should take an experienced programmer no more than 30-45 minutes to write a good solution, but it has room for ‘showing off’. We’ve had candidates spent half a day and delivering it with design documentation, unit tests and api documentation, but that’s up to them.

It is also designed to not only show common coding techniques, but also analytical and interpretational skills.

How the test is used in the recruiting process is different in some of our countries. In NL for example, it’s only after the first interview that we ask someone to take the test. In the UK, where we get many more CVs, we’ve moved the test in front of the first interview. In all of the cases however they are used as a preparation for the technical interview, where we discuss a candidate’s solution, why he made certain decisions, how he got to solving it in a particular way etc.

If a candidate doesn’t want to invest a bit of his time to do the test, that’s fine with us. However since we do invest in the time to get to know you (it takes us longer to analyse your code than it takes you to write it) and to make sure that there’s a good fit, we think it’s fair to ask a candidate to write a bit of code. Candidates that shop their CV at 15 different companies are not interesting to us. We want to see them as more than ‘just another developer’ only if they are willing to see us as more than ‘just another employer’.

Finally, the plumber example is very unsuitable. Neither is plumbing anywhere close to resembling coding, the terms of engagements are much different. If I were to go into a long term contract with a restaurant to supply us with lunch and diner, I definitely want to have eaten their food before I’d even consider it.

Elizabeth M Smith wrote at 11/18/2009 9:52 am:

Ivo – a 30-45 minute test is acceptable – a 5 hour to several day “coding test” is not

That’s like asking for free work

Anything that would take over an hour is just ridiculous

Carl Seleborg wrote at 11/18/2009 11:21 am:

Elizabeth, why on earth do you consider such tests as free work? Do you seriously expect a company to actually use a piece of code written by an unknown developer with a very vague specification?

5 hours of coding to complete an interesting test may very well be the best time investment a candidate can make if his or her objective is to get a good job with competent colleagues. What you can do, if you’re unsure whether it’s worth it or not, is simply to call the recruiter and gently ask how many candidates that return their test actually get hired. Anything over 5% and, granted, you’re wasting your time.

Chris Wenham wrote at 11/18/2009 8:13 pm:

Even the smallest firm should give a code test. I give candidates a few sheets of blank paper and ask them to do FizzBuzz and a recursive factorial, for example, and this is sufficient for typical IT programming positions.

Above IT there are software development firms, who should probably sit the candidate down in front of a workstation with a copy of the shop’s chosen compiler and ask them to create a basic “Address Book” or similar, which ought to take an hour or two.

Amazon, Google, Microsoft, IBM and others of that league will take a day or two of your time. If you pass, you’ve won the lottery. Congratulations. But it won’t be cheap to play.

Mr. Savage, when you’re offered the brass ring, I think you will prostrate yourself. I’m certain of it. If Pixar asked you apply for the Renderman team, I think I’d see you devote an 80-hour week to their test–even if they just asked you for Quicksort. Because the payoff is *that big*.

You do not seem to understand programming. It’s not a trade. Regardless of the language or platform, programmers become partners of the companies they work for, even if they don’t get stock options.

Michael Kimsal (@jsmag) wrote at 11/18/2009 8:30 pm:

Chris, I’m not sure “You do not seem to understand programming” is fair statement to make. In many situations, it *is* a trade, and is certainly seem as such by many many companies, both large and small. Whether it’s “correct” or not doesn’t matter.

I’ve worked in all sorts of companies large and small, and very few take the attitude that “programmers become partners”. Many pay lip service to that idea, but few execute and continue to hold to those principals. The ones that do tend to not just single out programmers, but have that philosophy pervade the company at all levels (which is nice).

For many people, too, programming is a trade. It’s a skill they have, the come to work, provide the service, and go home at the end of the day, nothing more. Personality-wise, it’s hard for me to be like that, but I’m growing to accept it in others, especially depending on the work environment.

Yes, in some rarified exceptions – the googles and apples and microsofts and such, a programming position with one of them can catapult your career in to a different stratosphere, much like (I’d think) attending a private school may put you in a rather unique world that few have access to (the contacts alone are what make many situations valuable long after you’ve left).

Few companies treat their programmers as ‘partners’ when it comes to making stategic decisions, for example. Yet those same strategic decisions will have a direct impact on the programmer, both in his/her direct duties, but often long-term as a deciding factor as to whether they still have a job or not (layoffs, company folding, etc).

The world of ‘programming’ is far too broad for blanket statements like “It’s not a trade”. And certainly the “you do not seem to understand programming” comment is in poor taste.

Tests can be an effective way to help screen some candidates, but the use of tests, and what kinds of tests, seems to vary just about as much as the people doing the hiring and interviewing. Just as each interviewee is unique, each interviewer is also unique, and the needs of both should be served during the process. If someone is insulted by *any* testing procedure, I might have some reservations about that person. On the flip side, I’ve been insulted by some of the tests I’ve been asked to do, mostly because it indicated the position was not what was advertised, or the imposition on my time was *far* too great for the potential payout. Not ever job interview is with Google or Pixar, and the people hiring need to have a realistic view of their own company and the value proposition they’re offering as well.

Chris Wenham wrote at 11/19/2009 6:01 pm:

Whether formally recognized as partners or not, a programmer becomes a partner even if the company doesn’t realize or even want it.

Programming means making decisions every day that will create or deny options for the company to grow later on. Even lowly grunts in an IT department can make choices that, months or years later, can make it possible for the company to take advantage of a new business opportunity, or make it impossible because the re-write would cause them to miss the opportunity and pass it to a competitor instead.

Decisions as seemingly trivial as “do I use MD5 or SHA?”

Computer programs are not products, they’re like shims and kleenex. Programs are a disposable artifact on the way to delivering the programmer’s knowledge of the problem and how to solve it. The doctor is the product, not the bandage. A business that has programmers have an asset, but it’s not the software. The software is a liability.

This applies even to commercial software, which you buy as if it were like a product. But with the exception of games, software stops being useful after only a few years, sometimes even a few months. It has to be rewritten, not just to support new operating systems and hardware but because the problem changes or our understanding of the problem changes.

And this is why programming isn’t a trade. Businesses who think it is and try to treat programmers as tradesmen do so at their own peril. If a company is using programmers for anything significant to the business at all, then the programmers will be co-inventing that business with them.

Some companies don’t realize this, and they don’t screen and interview their programmers properly. They set themselves up for a lot of problems when they do that, and not just when they get a “blub” who doesn’t want to learn anything sharper than Visual Basic or PHP. They will get programmers who don’t understand the company and what it needs, and they carry that company–invisibly, and silently–into the gutters with every line of code they write.

Ivo (@Ijansch) wrote at 11/20/2009 4:08 am:

I’ve already responded to this post earlier but it prompted me to be a bit more verbose about what our entire recruiting process looks like. I’ve posted it here: http://www.ibuildings.co.uk/blog/archives/1578-Its-not-a-job,-its-a-challenge.html

Hari K T (@harikt) wrote at 11/21/2009 1:52 pm:

Wow good post, great discussion.
And I agree with the @weierophinney interview :) technique.

The Heretic wrote at 1/24/2010 3:57 pm:

After my last position, I have decided I want to see the code of the people hiring me.

Never again do I want to work for someone so completely inept at coding and managing a project, but so good at talking a good game.

I don’t do well at impromptu coding on a white board and solving any non-trivial problem in less than a few minutes. I *can* solve most problems by thinking about them a little while, trying different things, googling to get ideas (or to just reuse well designed and tested code) – but I have to sit down to a computer, use an IDE and work at it.

The chance that an interviewer will ask me a question that I can solve easily on a whiteboard in just a minute or two while under the stress of an interview, is nill. I view such challenges as more of a passage of rite by the interviewer – they had to do it, and they’ll be damned if they let you get by without having to do it too. The idea that they have the insight to be able to see how you solve a problem from such a test is pure hubris at best.

Ask me for some code samples, I will be glad to share them and discuss them. Ask me, during an interview, to write code on a whiteboard for a problem I haven’t thought about and I will probably choke on it.

Phil (@zoldello) wrote at 2/2/2010 5:44 pm:

For a first round- yeah it is as horrible as writing a 2,000-line method without comments (bad on many levels). However, for a second round interview or the last-step-before-hire; I think it is acceptable. Just because someone has “senior level” on their resume does not mean they can do “senior-level” work. A thorough test like this can prevent an employer from having to go through the hiring process again just because they go a non-coding sweet-talker on their hands.

chris reiss wrote at 2/19/2010 4:55 pm:

This may be a little zen, but ask the person to come up with their own problem, and have them walk *you* through it.

You want to hire people smarter than you. If they are, they can ask better questions. Let them.

Copyright © 2023 by Brandon Savage. All rights reserved.