Painless Spec and Schedule Development

In the time that I have developed software, I don’t know that I’ve ever met a developer who got excited about writing specs for anything. In fact, most developers loathe writing specs, or developing schedules of any kind. It’s not that they’re lazy, or that they don’t want to be held accountable; most of the time it’s because developers prefer to express themselves via code, or because developers are afraid that if they set a schedule, and then reality doesn’t match up, they’ll be forced to produce sub-standard code. Neither of these is an ideal situation.

This is directly at odds with the business need of specifications and schedules. Businesses need schedules to know when products will be finished and schedule things like trade shows, product launches, and write contracts with clients who need or want a particular product. It’s not as if businesses want to push their developers to insanity by forcing them to schedule and then stick to it; more often than not thousands of dollars hinges on the schedule, and it simply must be met.

Schedules and specs are a core component of software development, and business development; so much so that Joel Spolsky included developing both as core components of the Joel Test. While developers hate writing specs and developing schedules, there are some painless steps they can take to create them.

Specs need not be complete documents
Lots of times, specs need not be complete documents that contain specific, detailed information. In fact, lots of times specs need not contain a single complete sentence to be effective. Specs are, in their most basic form, a description of the way that something should work. A wire frame, if done properly, is an adequate spec. The workflow of an application together with markers, is a spec. My specs are simply drawings of the view that I want together with some notes, arrows, and perhaps some writing on the back; this works well because it describes the way the application should look, and the rest of the spec contains database diagrams and other views to enhance understanding.

Writing a spec doesn’t actually mean opening up Microsoft Word each and every time. And the client shouldn’t be expected to generate a document like that, either (see the third section below). Specs can be representations of functional apps.

Specs are not documentation
There seems to be another push in many groups to be able to use the spec as documentation later on. Don’t do this! Specs are not documentation. They are descriptions of how the thing should work, not diagrams for how it does work.

Actually implementing the details of the spec may result in a disconnect between the implementation specified and the implementation completed. This is normal; programmers often discover logical mistakes in specs when they actually go to implement them, and must make adjustments. This occurs in every field; engineers, for example, must sometimes adjust their underground services to route around unmarked power or sewer lines. Deviation from the spec is normal, as long as the end product looks like the spec.

However, if the spec is meant to be the documentation, and there is no time in the schedule for writing documentation, there’s intense pressure to make the spec and the implementation look identical. This leads to unusable or difficult software, or worse, bugs. Spend time to write documentation, rather than relying on the spec to do double duty.

The development of the spec should be done by the developers, not the client
There’s an old customer service adage: The customer is always right! This is not true, however, when it comes to the details in the spec.

Consider: when you hire an architect, you tell him that you want a house. You do not, however, hand him the blueprints. Instead, he draws up plans and works with you to develop the blueprints for your house. Handing him a set of blueprints and telling him to build the house would eliminate the need to have the architect in the first place! The same is true for the development of specifications.

Developers should work with the clients to make sure that the needs of the client (“I need an invoicing system” or “I need a new website with a blog”) are translated into technical specs (“We need to use Zend_Pdf with Symfony” or “Drupal is the best option here”). The client that comes in and insists on building an invoicing system with Drupal is NOT right.

Developers should be experts in solving technical problems, and therefore should write the specs. They should work with the clients to find out the needs, and developing specs in this fashion can actually be one of the most rewarding parts of software development.

Schedules aren’t all about developers
Moving on to a discussion about scheduling, it’s crucial to note that schedules shouldn’t be all about the development. In fact, a giant chunk of the total time it takes to move a project from spec to deliverable doesn’t have anything to do with programmers. Why then, does it seem like the programming team is the group that always gets stuck writing a schedule?

The programming team is often asked to write the schedule because management mistakenly believes that programming takes the most time (more on that in the next section). However, there are large parts of the project that have nothing to do with programming: the client taking two extra days to review the demo, people getting sick on the QA team, management mistakes, vacations, etc.

Developers make up a small, but crucial part of the development calendar. When schedules are developed, they should be considered but not weighed as the only factor. Managers must learn to take into account other factors.

Some basic rules that I live by when writing a schedule: December is only 50% productive. November to February, 1 person is out sick every day. June to August two people are on vacation at any given time. These things must be taken into account; otherwise, the schedule will be off considerably.

Writing code is only 1/6th of the total development time
According to Frederick Brooks in The Mythical Man-Month, he has “…been successfully using the following rule of thumb for scheduling a software task: 1/3 planning, 1/6th coding, 1/4th component test, and early system test, 1/4th system test, all components in hand.” This is considerably different than the way most software development tasks are scheduled – in fact, a full 84% of the scheduled time doesn’t involve coding!

Wait, what?

That’s right. A full 84% of the total scheduled time will be spent on tasks other than programming. This is an important concept, because it completely breaks apart the conventional wisdom of managers. They think “software developers should spend their time developing software! And software is developed by lines of code.” But this line of thinking is sadly wrong.

Programmers sometimes take issue with this as well, because they also see their role as writing lines of code. But software isn’t developed by lines of code. LOC is the ultimate end product; however, there is considerable architecture, design, and testing that goes into a software product. Believing that lines of code make software is like believing that plastic makes a toy. It’s a component, but not the whole thing.

For developers, when we estimate we need to consider that our estimate likely only considers the time to write the code, and that’s when things go right. We need to fully inflate our estimate by 84% to account for the extra time it will take to develop the application.

Evidence-based Scheduling Isn’t Optional For On-Time Products
Anyone who has ever experienced a schedule slipping and the stress that ensues probably realizes that they never want to experience this ever again. But how does a development team accomplish this? By scheduling based on evidence.

Evidence-based scheduling is simply the idea that the schedule should be based on hard facts from history, rather than the best guess of the developer at the time. The more estimates and schedules tracked, the better the evidence used to predict the schedule.

For example, say that Programmer A is always 25% short on his estimates, and programmer B is always 30% faster on her estimates. If Programmer A estimates 1 hour, it will actually take them 1.25 hours; if Programmer B estimates 10 hours, it will actually take 7 hours. With this data, it’s possible to predict more accurately when a product will be finished, not just based on the programmer’s best guess, but on actual hard facts that relate to their best guesses.

Lots of software products support evidence-based scheduling. FogBugz supports this by default; Jira has a plugin (which you must pay extra to obtain). These products make it easier to predict how long tasks will take, and create more accurate schedules.

Brandon Savage is the author of Mastering Object Oriented PHP and Practical Design Patterns in PHP

chris (@enygma) wrote at 12/16/2009 7:47 am:

I have to disagree with you on one thing – you mention that the spec itself should be an overview of the functionality, the “why” it should work but not the “how”. Putting the spec writing in the hands of the developers is a sure way to miss what the client is trying to tell you they need. Should it be all in the client’s hands? Of course not, but it should be a collaboration between them, not a one-sided look at how one group thinks it should work.

In my experience, the more vauge the spec, the less happy the customers end up being (and the more the developers have to come back and ask questions). The better specs I’ve seen are combinations of both narrative and process flows, giving the developers a good overall view (the flows) and some of the more picky details about the parts of it. I see it as the role of the project manager and/or a business analyst type to handle any communication between the customer and the development team. Last time I checked, most developers don’t have the business knowledge to be sure the spec is still well defined – especially if it’s an outside company they’re doing the work for.

Brandon Savage (@brandonsavage) wrote at 12/16/2009 10:27 am:

Chris, if I didn’t explain very well the collaboration I apologize. That was the point I was trying to make with the architect example; the two have to work together. The spec should also be specific, but as you’ve probably learned, specific doesn’t necessarily mean written in sentence form.

I like the idea of a technical project manager who knows his developers and acts as an intermediary between the two. Letting the developers do the development is the best option. It’s unfortunately also been my experience far too often that project managers are not technical, which creates significant problems for developers who actually do the development.

Benoit wrote at 12/17/2009 3:36 am:

Hi,

that’s the reason why we souhld separate functionnal spec from technical specs.
I know it’s no more sustainable having a too heavy software design upfront the production of code, but in fact it sometines still being required. Functionnals spec determines the what software does, like in use cases, while technical specs states how the functionnality will be done.
We’ve sometimes to wrote only to avoid the client discussing technologies and design choice we’ve made from the beginning of developement.

I agree on fact that developers should participate in writing specs. I ‘m used to do it (and liked it), but our management has made the choice to hire “professional” project manager in charge of writing them. That’s a disaster on term of planning, and profit, because she doesn’t have any idea how complex it is to make a functionnality in the real world. Moreover she writes things that are technically not possible, like putting flash app in email newsletters…

In the end, our clients ended up frustated because the specs are not filled, and we have very limited profits because of the large difference between time planned, and charged to cusstomer, and the real time we’ve to spend on the product.

Yeah, it’s a project manager job handle communication and perhaps negociation with client. But this is not incompatible with developers actually writing specs, or at least review and amendate them.

In my opinion, the more realistic and precise specs are, the more probability for the project to be a success for us and client.

Matthew Purdon (@mpurdon) wrote at 1/2/2010 9:47 pm:

I think a lot of the problem with the waterfall method of software development is the fact that you do rely on having extremely detailed specs in place before you can write anything. Although I agree that you definitely need good specs,I have found many times that the client really doesn’t know what they want. If you go ahead and implement something and let the client play with it as soon as possible, you waste less time grinding out specs that are just going to be tossed out later and have an easier time attaining the YAGNI principal. Long story short, get a good idea of the feature the client wants – especially what it is supposed to help the user do – and then bang it out. Clients are happy to get results back in small increments, it makes them feel as though their money is actually going somewhere.

Copyright © 2023 by Brandon Savage. All rights reserved.