| « Why Every Developer Should Write Their Own Framework | Why Coding Tests Are A Bad Interview Technique » |
Nearly a decade ago, Joel Spolsky came up with a method by which to evaluate software development shops that has come to be known as the Joel Test. This crucial test evaluates a software development company on the basis of twelve criteria points; Spoksly said that “a score of 12 is perfect, 11 is tolerable, but 10 or lower and you’ve got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time.”
When Joel wrote the test, there wasn’t much development for the web; the little that was being done wasn’t being done in any of the modern languages that we write in today. In fact, Facebook, Myspace, Twitter, Gmail, and LinkedIn hadn’t even been invented yet.
Today’s world makes heavy use of web-based software (the term “software as as a service” keeps floating around). And so, it is necessary to update Joel’s test in order to properly apply it to web development. This has been done by some folks, but I will do it again, mostly because I disagree with them.
Note: I will not be reprinting the original Joel Test; you can read that on your own.
The Joel Test – 2009 Edition
Again, just like the original, you can answer a quick yes or no to these questions in about five minutes. You should know how your operation scores without doing much deep thinking.
Let’s look at each point separately.
1. Do you use source control?
This is completely unchanged from the original test, because it is one of the most crucial components of professional software development. Whether you’re a team of one or one hundred, you need source control. Since 2000, lots of new options have been introduced: Mercurial, Git, Subversion, to name a few. If you’re not running version control you may as well hang a shingle that says “your code is not safe with us.” Because it isn’t.
2. Can you make a shippable version of your software in one step?
There’s a lot of argument about how or whether this applies to web development. It does.
Web developers need to have a build process in place. That build process needs to do things like run the unit tests, create a tarball or package version for distribution, and run other tasks. My personal build process builds a manifest of all the classes for the autoloader and strips comments out of my code.
For PHP, we have a number of tools available to us. My personal preference is Phing, but that’s only because I’ve used it; some prefer Apache Ant. Whichever you use, you need a build system.
Someone will inevitably argue that the nature of the web allows for deployment of code without creating a build. This is true, but it also lends itself to bad development practices. If you simply do an “svn up” whenever you’re ready to release new features, you never actually go through the process of reviewing, tagging and releasing something; you’re always updating from trunk, and this makes it hard to version and fix bugs. Yes, there is some overhead incurred, but this is necessary to the art of software development.
Do you use continuous integration?
The time to find out that one of your developers broke the build or failed a unit test is within hours of their commit, not at 2 am in the morning on the day after the release was due at 5 pm.
Continuous integration (there’s phpUnderControl and XincM for PHP projects) allows you to regularly build your application, test your application, and write the documentation for your application. If the tests or the build fails it can alert you to problems before you’re ready to roll a release. Use it. It works well, it’s easy to set up, and it’s invaluable.
Do you have a bug database?
This point also remains unchanged, due to the fundamental necessity to software development. There are hundreds of bug tracking software packages that will integrate with Subversion or other VCS programs.
Tracking bugs provides the ability to see what issues remain, who is working on them, and why they have yet to be resolved. It also allows for the measurement of metrics (e.g. knowing how long the average ticket is open). Tracking bugs is crucial, because if you ship a product repeatedly with the same bug, your customers will get antsy and frustrated. “Oh, we forgot about that one” stops working after using it for the third time.
5. Do you fix bugs before writing new code?
When deciding how to work on a release, you should always fix existing bugs before adding new functionality. Always.
This might seem counter-intuitive but consider: your customers will be more annoyed by persistent bugs that hurt their ability to use your software than they will by a lacking function that they want you to integrate.
Consider the reality that a customer who sees a bug automatically has a lower level of trust in an application. If they’ve reported it and the bug persists between versions, that trust will be even lower. This is certainly true of me: the WordPress cron task has been broken since WordPress 2.7.0 and remains broken; I have less trust in WordPress as a result.
Additionally, the longer you wait to fix a bug the more expensive it will be. That bug may affect six features (all of which you added in the last version) instead of one, necessitating a complete rewrite of all six features (and requiring you to do the work twice). Don’t do that to yourself.
6. Do you have an up-to-date schedule?
A schedule is critical for business strategy and planning. But it’s also crucial for keeping the programmers from being surprised. Developers don’t like surprises. And neither do business people.
A schedule is never set in stone; it must be fluid and respond to the needs of the development process. For example, if the schedule is slipping there are really two choices: relax the schedule or reduce the requirements. By having a schedule you avoid scope creep, because items not on the schedule can safely be ignored. And having a schedule that is up-to-date means that everyone knows when, how and where the deliverables are, at any given point.
6. Do you have a spec?
Joel Spolsky spends a good deal of time writing about specs in a four-part series. Apparently when people see “spec” they think of 600-page documents that no one ever reads.
Bzzzt. Wrong.
A spec is as simple as a wireframe that describes the functionality and the views that the application is going to have. A spec is as simple as defining the scope of a feature. A spec is as simple as a bullet-point list that describes how an application will work.
Specs allow you to plan. They allow you to more accurately estimate. They allow you to ensure that the customer is getting what they want. They allow you to ensure that all the points have been hit before you start writing code. Before you start writing code. That’s the big point: specs must be done before code is written.
By putting a spec on paper, you also avoid feature creep and last-minute adjustments by the client. They’ve signed a piece of paper saying “yes, these are all of our needs for this version” which you can then take back to them, with a promise that you’ll integrate their request in the next iteration. Your developers don’t get angry at you for springing a new component (and lots of overtime) on them at the last minute, and your clients stay happy because they get what they ask for (even if they have to wait longer when they change their minds).
8. Do programmers have quiet working conditions?
This Joel Test point is often contentious amongst those who think that a big, collaborative atmosphere is best. Joel argues that developers should get their own offices. Having worked in both situations, I agree with Joel. Developers should work in offices, not in cubicles or big spaces.
In order to program effectively, you have to be able to hold lots of data in your head at the same time. If you’re interrupted, that “RAM” gets cleared and you start over from scratch. I’ve lost many a day because I just couldn’t get in the zone: there were meetings, phone calls, emails to reply to, and other issues to resolve.
Programmers are expensive, and they need to be treated like any precious resource. Place them in their own space, which they can decorate, and give them a real door that shuts. Give them the ability to shut out the world while they work. The code will be better for it, as will the end product.
9. Do you use the best tools, and spend money on those tools that are necessary and expensive?
When the Joel Test was written, many of the open source languages we use weren’t popular and some didn’t even exist. But they exist now and this point should be modified to reflect that.
When open source tools are available, and they are the best, you should by all means make use of them. Many languages, applications and platforms are available for free through the open source community, and they are just as good or, in some cases, better than the paid tools you can buy (feature set for PHP versus ASP or .NET, anyone?).
But when it comes to paid tools that are both necessary and expensive, are you willing to shell out the cash? Do you buy your developers awesome machines? Do you cater lunch for them? Do you take care of them with software and books and tools? If you don’t or are unwilling to spend the money to do so, this is a problem.
10. Do you write unit tests?
In order for tools like continuous integration and a build system to be effective, you must unit test your code. Unit tests are an effective way to ensure that your code is accurate, that refactoring doesn’t break the expected results, and that your release is ready to go.
Lots of projects don’t use unit tests. But more projects should. Writing unit tests is a critical component of identifying early on that something is wrong. In fact, one development methodology called Test Driven Development relies solely on writing tests first, before any other code is written.
Unit tests are necessary to ensuring the integrity of your application.
11. Do new candidates demonstrate their code knowledge in some tangible way during the job interview?
I opted to change this point due to the fact that some people were asking too much in interviews.
You want to ask candidates to demonstrate their code knowledge – either by writing some simple functions or discussing programming concepts and topics with you. You might show them some code and have them debug it; this is also a great way to find out if a candidate is qualified.
Try not to make this into too much of a quiz scenario; coding knowledge is displayed in logic, thought process, and architecture skills, not in the memorization of function argument orders and obscure data. The general rule of thumb is that if you can find it in the documentation, don’t ask about it in an interview. Google is a great resource when on the job.
12. Do you have testers?
Why have I dropped hallway testing and opted for testers instead? The reason for me is simple: in an excellent development shop there is an entire team devoted to UI, who should be responsible for the data getting into the system and formatting it when it comes out. The programmers should very rarely interface with the UI team’s work.
This is not to say that developers should have no user experience training. They should be competent enough to chime in on the discussions and potentially work with the user interface if necessary. But the hallway testing should take place on the UI team side of the house.
Testers are another matter altogether. They are required for both UI and development. Someone needs to bang on code and figure out how it’s broken, and send it back for fixing. The testing department can be shared with the UI team and the programmers, but should be independent and able to veto any release of an application that doesn’t meet the spec and function well.
Summary
This update of the Joel Test for the modern web world should serve you well when looking for a new job or rating your existing shop. It integrates the most important components of the original test while adding new components and deleting dated ones, as well as adapting the test to a web-centric development process. Good luck!
Brandon Savage is the author of Mastering Object Oriented PHP and Practical Design Patterns in PHP
Posted on 11/4/2009 at 1:00 am
Giorgio Sironi (@giorgiosironi) wrote at 11/4/2009 2:29 am:
Unit testing is becoming fundamental: Unble Bob once said that it’s like double-entry bookkeeping for accountants or washing hands for surgeons. In open source projects like Zend Framework it’s the test suite that allows users to contribute with code and patches.
RobTW wrote at 11/4/2009 3:37 am:
Great article. Especially the spec part inspires me a lot.
Michael wrote at 11/4/2009 5:56 am:
Great post on Joel Test To Web Development. I have compiled some useful inforamtion on PHP proramming skill set and development platform with some intresting case studies. Hope it makes useful reading for php programmers at http://www.angleritech.com/technology/outsource-php-development.html
Ryan (@popthestack) wrote at 11/4/2009 10:48 am:
The next time I look for a job I’m using your new and improved Joel Test. I’ve tried using the Joel Test in the past, but as you have pointed out, much of the questions didn’t work.
Sandy Smith (@SandyS1) wrote at 11/4/2009 12:28 pm:
You can also use “svn switch” and switch from release tag to release tag if you don’t want to treat trunk as the “stable” branch. Or you can have a “production” branch that you merge into and test before deployment. It actually works very well.
Brandon Savage (@brandonsavage) wrote at 11/4/2009 12:31 pm:
That technique does work, but I prefer not to have the hidden .svn files in my document root. While you can have Apache refuse to serve those files, often people do not, and if you’re deploying the code on a client’s server, you probably don’t want to give them access to your subversion repositories.
Erik Snoeijs (@esnoeijs) wrote at 11/4/2009 1:52 pm:
@Sandy
“best” way for deployment using svn (very much IMHO), is by using svn export. (leaves out all those pesky .svn files)In a script you can simply export your new release in a shadow directory. Do all the other magic that needs to be done and then
mv live live-04-11-09 && mv shadow live;
Joseph Scott (@josephscott) wrote at 11/4/2009 2:39 pm:
Wondering where your patch is with detailed bug report and tests for the wp-cron issue you linked to – http://wordpress.org/support/topic/227742?replies=6
My patch was to revert to the WordPress 1.6 cron script. I’ve got six OSS projects of my own (one of the reasons I use WordPress at all). I don’t have time to fix their buggy software.
Perhaps the “Always” rule for fixing bugs before writing new code isn’t written in stone after all :-)
Well, since I’m not on the WordPress team, I’m not bound to fix their bugs before I work on my own code.
However, in my own projects, and in projects that I do for employment, I always fix code before releasing new code.
Jacob Santos (@jacobsantos) wrote at 11/4/2009 4:39 pm:
Well, I think the problem is that the code (wp-cron) has been “fixed” almost every release. The problem is that people have different problems that come up. Lately, I believe you have the ability to just call that page directly using lynx and crontab.
pbg (@pbgswd) wrote at 11/4/2009 7:09 pm:
Hi Brandon, I really have been enjoying what you have been writing about lately. You point out a very important part of the software industry, especially in our field, that is in dire need of some updating. It is better to inverse the process as you are suggesting by asking the employer what they have in place for YOU. Its even more important, I think, to out flaky employers, clients, projects, because developers can be used, wasted, and exploited in poor quality environments.
It really seems that for our time the Joel test is broken. Also, I have never found an employment situation that was actually superior to the working environment I create for myself. If you arent finding employers that measure up to your standards, or ones that don’t know how to evaluate candidates, perhaps it is a hint that you should be running your own business with a team of peers instead of selling off your future to some company. Why do you want to work for other people when you are smarter than they are?
Ivo (@Ijansch) wrote at 11/5/2009 4:01 am:
If you’re interested in PHP and the Joel Test, you might also enjoy this presentation by Lorna Mitchell:
http://www.slideshare.net/lornajane/passing-the-joel-test-in-the-php-world
Fake51 wrote at 11/6/2009 8:45 am:
I agree with most of the points on the list and find it a good tool to judge yourself (as well as others) by. There are a couple of things I’d be less absolute about.
The build process can easily, as has been stated before, be handled via SVN. It’s actually rather easy to setup your SVN server with access handling so that given clients can only read from your repo, not write to it. Which means that if you don’t update it, they won’t have access to anything they wouldn’t have access to otherwise. And you keep the ease-of-use that is the svn update.
Also, there are times when clients will be screaming at you for a given feature. Telling them that bugs must be fixed first is not the best way to handle the issue. I guess the difference here is between working continuously for the same client and then shipping new products to existing/new clients: if you have the luxury of spending time between versions working on your product, then by all means, fix those bugs – but if you’re working a site for a client, and they all of a sudden need that wicked, new feature to promote something or other, then get that implemented instead of trying to fix some bug in the backend. Otherwise you’ll see your clients looking for someone that handles their priorities better.
Btw, how does catering lunch equal paying for necessary tools? Just wondering :)
| « Why Every Developer Should Write Their Own Framework | Why Coding Tests Are A Bad Interview Technique » |