Knowing the scope of the problem is more important than knowing how to solve the problem, because you can try many experiments on your way to figuring out how to solve the problem, but if you never really knew the problem in the first place then all of your experiments will be for naught.

 So you’ve just been given your first programming assignment, and you’re wondering where to start first? The good news is that Step 1 is the time to relax, grab a cup of coffee, settle down for some quality time with the spec (or this tutorial) and the clientèle, and stay the hell away from a compiler. Never jump into writing code first because you will create a heap of rubbish that’ll require effort to clean up. The very first thing you should do is write down your idea of the problem’s scope as you think it is, then go out and see just how wrong you were.

 Do this by composing an email and sending it to the people who commissioned the project and who will be affected by it¹(known as the principals or stakeholders). The email will begin something like this:

Dear Principals,
  The Plonk Reprocessing Manager will reprocess plonk by diverting 12% each of the Merlot, Cabernet and Shiraz as they go to caskets for ageing, combine them into a 5,000 gallon steel tank and spike it all with cheap brandy to increase the alcohol content. The PRM will have controls for selecting the exact percentage of pre-aged wine from each source, as well as the amount of spiking. PRM will have a Web Browser interface and will automatically send email to the master vintner whenever the mixing proportions are changed by any user

.

 You will get no replies except “K, thx”, but now you’re on record. Next, you print out this email and head out on foot to visit the principals where you will buttonhole them and read the statement-of-scope out loud. At this point, you may even deliberately change something to see if they are listening. EG: “The PRM will reprocess the plonk by draining caskets of 10-year aged Cabernet Sauvignon…” They should stop you and spend an hour re-explaining how everything works, and it’s that hour of deep discussion that you really want out of them.

 Do this for as many people as you have the patience for, and carry around a clipboard to take notes because you’re going to learn a lot that wasn’t in the original proposal. 

 If the program is automating a manual process then you should try–or petition for the right to try performing that process yourself so that you can become what the program is supposed to become. At the very least you should spend time watching people do things the old way, talk to them, ask questions, let them show you pictures of their kids, buy them drinks, get them drunk and provoke them into talking shop.

 And I’m really not kidding about getting the principals drunk or talking about their kids, because they will loosen up and start telling you about all the things that annoy them and make their life difficult. Getting to know and understand the scope of the problem is the most important step and the messiest; it doesn’t lend itself to clean-room techniques, top-down approaches, focus groups or questionnaires. If you do not really know what the program is needed for, then whatever you build will never be used. 

 The end-product of this step would be a functional spec and some use cases. A functional spec is a short document (1 or 2 pages) that describes what problem the software will solve and what it will do to solve it. Use cases are step-by-step walkthroughs of what a user will click and type and what they will see in return. A typical use case for logging-in might look like this:

1. User clicks on shortcut to PRM web console
2. Browser loads login page
3. User clicks on “Username” field and types in her user name
4. User clicks on “Password” field and types in her password
5. User clicks on “Login”
6. If the username existed and the password was correct, the PRM control screen is displayed

 These will be important to show the principals how the software is going to behave to them, and for when you need to delegate part of the project to another programmer.

1 – If you’re writing mass market software, you or your company should recruit some volunteers who are in your target audience.

Step 2: Fix anything that you know will give you a headache

 Even before choosing your tools you should fix everything that’s within your power and that you know will interfere with the job. Such as:

1. “Broken windows”–bugs in other programs you maintain that keep interrupting you for manual fixes. Fix them properly so you won’t be distracted
2. Missing specs or documentation that you’ve been promised to get “Real Soon Now”. Put the project on pause and hound the sources for that material so you don’t find yourself forced to change your design halfway through the project
3. Problem co-workers and clients–Inform them that you’re “On Project” now and they need to go through someone else for their tech fix. Tell them that you want to give them your undivided attention after you have completed your assignment. If you have a manager then it should be his job to keep these people away from you, so remind him if necessary
4. Clean up your filesystems, desk, inbox and coffee cup

 And even if you don’t think there’s anything to do, take a day to look anyway. That’s because the secret, hidden agenda of Step 2 is to get your mind off the project and make you forget some of the details. Not kidding this time, either, for this is a mind hack that leads us to:

Step 3: Review the problem scope again

 By taking your mind off the assignment for a while in Step 2 you want to put yourself in a position to see the assignment again from afresh, or as fresh as you can get. These gaps in our focus are called creative pause, and they can help you see details you may have missed because you were focusing too hard on whatever you were debating with your coworkers about in the break room. Step 2’s hidden goal was to get you away from the trees for a moment so that you’d see the forest upon return.

 If your review compels you to change the definition of the problem or function of your program, then edit the spec to reflect this.

Step 4: Whiteboard the problem and reduce, reduce, reduce

 Bullet-point everything about the problem on a whiteboard, then start erasing anything that’s irrelevant or doesn’t need to be solved with a new code. EG:

• Templating system for web interface <— Use ASP.Net/RoR/Django
• Authentication system <– .htaccess files?
• API to control spigots
• API for tank level sensors
• Install SMTP server to send mail <— Use mail.companyname.com
• SQL database to store mixing levels <— Flat CSV file
• Feedback control loop
• Machine Learning algorithm <– WTF?
• Splash screen
• “About” screen
• Logging system <– Log4j/NLog/Log4net/etc
• Help file
• Documentation <– POD? Sandcastle? HTML?
• Command-Line Interface
• Easter Egg

 This is your first opportunity to See If Someone Else Already Did It, because it’s only by this stage that you could consider yourself an expert in what needs to be done and would know an existing solution if it snuck-up and kicked you in the bollocks. You might have wanted to blurt out “but we could do this with an Excel macro!” when they first started to explain the problem to you, but this would have been wrong. This is why I told you to stay the hell away from a compiler in Step 1. Step 4 is when you go to Google and look for people who have already solved the same problem in whole or in part and evaluate their software to see if its fits your need.

 Then be ruthless and kill anything that you can’t solve by other means or that you don’t really need. “A designer knows he is finished not when there is nothing more to add, but when there is nothing left to take away.”

 Already at this stage you may have started to discover tools for modules based on what lends itself to the job. So now it’s time for:

Step 5: Choose your tool chain

 A tool chain is going to be your platform, language, compiler, framework and libraries, IDE and design patterns. Each will influence the choice of the others, but the language you’re most familiar with should be the strongest influence of them all. There is a time and place to learn new languages, and this project might even be one of them, but think about it very carefully: what’s the project worth to you? Is it going to determine if you keep your job or not? If this is truly your first assignment then stick to what you know, even if you think another toolchain has magic pixie dust that would be just sweet for this assignment. 

 If, however, there is a toolchain out there called “Language For Writing Plonk Reprocessing Managers”, then the risk is probably worth it. Certain classes of problems do have tool-chains built specifically for solving them, (Postscript, CICS, etc.) and this should have been something you dug-up in Step 4. If you find one then by all means take the time to learn it. It will be easier to justify to your employers than not using it, especially if they find out that you knew about it.

 The next most important factor for choosing your tool chain should be the ecosystem made up of third-party libraries and frameworks. This is where languages like Perl and Java get their strength; there is a ridiculous amount of third-party support that might solve whole chunks of your problem for you.

 This is also the stage when you’ll be doing your first design work, because it will be part of evaluating the tool. You should be thinking about how you’re going to build your modules with these tools and checking the documentation for the things you believe you’ll need. Like: do you need arbitrary precision math? Does it let you address the hardware serial ports? Does it support SSL? Do you need unmanaged pointers? Is there a device driver for this peripheral? And so-on.

 Some recommend sticking with the shop’s standard tool-chain (Microsoft shop, LAMP shop, Java shop, etc), but I don’t. It obviously makes sense to stick with what everybody knows so that employees can be swapped between projects, but this benefit does not necessarily trump all others. Some organizations like Amazon.com have a heterogeneous environment and do very well. They begin by defining an interface that the project should implement and then give the team the power to chose their own tools. 

 When you begin Step 5 you should plan to spend a few days for a small project (where delivery is expected in a month), 1 to 2 weeks for a medium sized project (3-6 months to delivery), and at least a month for very big projects (1 or 2 year delivery). Mark this down in your calendar and commit to using whatever tool-chain was the leading contender by the time the deadline arrives.

Step 6: Review the problem scope again

 Now that you’ve chosen your hammer, it’s time to have another look at the nail. This is because every project is about continuously oscillating back-and-forth between understanding the problem and inventing the solution. Each time you take a small step forward with one it changes how you see the other. This will continue even as you begin coding.

 This is also why projects get stuck in a never-ending planning phase, where new tools and abstractions make the developers rush back to the drawing board to sketch out how much better its gonna be when they upgrade to Ruby On Rails 2.4, which had just come out while they were developing a solution that was to be built on Django 1.1 or ASP.Net 3.5. This is why you need to decide in advance how long you’re going to spend on Step 5 and stick to your decision. There will never be anything so sexy about a new tool that trumps being able to deliver a solution on time.

 And don’t forget to edit the spec to reflect any changes in how you see the problem or the solution’s function.

Step 7: Sketch a plan of attack

 This is the last step before you begin coding. Here is where you’ll sketch out your database schema–if any–plus a basic outline to organize your code and a flow-chart of any crucial algorithms. You will not need to define every class and flowchart every method, just the ones that contain the business logic. You will probably not need to play around with UML diagrams.

 There are really only two approaches to software development: Big Design Up Front (BDUF) and Everything Else. If you deliberately make Step 7 longer than 7-to-10% of your delivery estimate then you’re doing BDUF, and the consequence is that you’ll either need to use a lot of code generators or commit to maintaining two separate definitions of the same program (because the design will change). You will know you’ve reached the tipping point when your code is more disposable than your design documents.

 The author’s philosophy is that coding is design and that Step 7 will get shorter as the programming language gets to be more expressive. Code files are design documents and the compiler is the code generator. But there will always be a Step 7 in every project because programming languages will never be expressive enough. You may even have a mandate to spend more time in Step 7 if the client requires it. This is true for government contracting, for example, because they require design documents and UML diagrams as part of the deliverables.

Step 8: Code

 Now there is nothing left to do but what you were ostensibly hired to do, which is to code. If you’re leading a team then, by now, your spec and plan-of-attack would give you what you need to delegate tasks, make a schedule and give time estimates. This is also when you may adopt some techniques such as burn-down charts, daily meetings, bug tracking systems and unit tests. 

 As you code you will find that your understanding of the problem’s scope continues to evolve and change. It isn’t unusual to get halfway through coding and have a head-slapping moment when you realize there was a much simpler way of solving the real problem. But unless the epiphany completely obviates the need for software, you must continue to completion and deliver the product, because it will probably still be the best way to make use of resources.

Knowing the scope of the problem is more important than having what looks like a solution to the problem, and sometimes going through the effort of building a possible solution is the only way to truly understand the scope.

Memo to staff,

Although costs have fallen 60% since the deployment of the new Plonk Reprocessing Manager last year, profitability remains elusive. Finally we know why. A report published by DWI (Drunkards and Winos Institute) this month indicates that buyers of plonk really just want to get sloshed, and that any old booze will do the trick. Our mission to provide higher quality plonk by combining multiple wine varieties was misguided, and we really should have just been bottling whatever we could press out of the cheap grapes we’ve been passing up all this time. Selling high-end plonk to drunkards was like selling gourmet ramen to college students. 

Effective immediately, the Plonk Reprocessing programme will be halted and all resources diverted to the manufacture of Grade C blush wine pressed from nasty-ass concord grapes. 

Regards,

The Management

Stagger Home Winery, Inc.