24

As a software engineer, we are always eager to get effective tools to boost our productivity. And in our daily work, we are often unsatisfactory with the existing tools and would like to have better ways such as better GDB script config, Vim script and some Python script to make boring things automatic.

However, it is actually a trade-off since making tools also needs time and energy. It does not give productivity boosting immediately. Therefore, how do you judge whether it is time to stop work and to make some tools to ease your future pain?

4
  • 30
    ObXKCD - Is It Worth the Time
    – user40980
    Jul 13, 2013 at 4:40
  • 1
    maybe refactor towards a tool and keep working
    – Ray Tayek
    Jul 14, 2013 at 6:45
  • 1
    @MichaelT Also this: geeksaresexy.net/2012/01/05/geeks-vs-non-geeks-pic
    – alroc
    Jul 19, 2013 at 17:22
  • 1
    That XKCD pretty much nails it except for one thing: whether the time saved by the tool is just your own, or whether it is multiplied by a large user base throughout your organization or beyond.
    – Kaz
    Jul 19, 2013 at 22:06

7 Answers 7

28

I "make tools" when one of these is true:

  1. The task is annoying me
  2. The risk of human error in the task is too big

The "risk" for the 2nd option doesn't have to be huge - the cost of building one small tool is usually small, so if all you save is the risk of running a 10-minute build again once a week, it will usually repay itself very fast.

I try to make tools as small as possible - just make the task a little bit easier now, and maybe improve again next time.

This means you only fix the biggest pain each time, and not creating fixes to problems that don't really hurt you.

2
  • 2
    +1 for avoiding errors because this is more than the usual time spend versus time saved when executing.
    – k3b
    Jul 13, 2013 at 7:49
  • 1
    I would add "when you find yourself repeating the same task very often". I, surprisingly enough, find myself needing to generate a random string of characters quite often. So instead of writing a blip of code to do it, I made a simple form with a button to do it for me
    – bsara
    Aug 27, 2013 at 15:21
9

With experience I have found that just pushing hard on the grunt work is usually the most time-efficient. Making a tool is often tempting. I give up resisting when:

  • The tool has more than one purpose. A good chess player accomplished two things with each move: block an opponents piece and free up a bishop. A beginner would probably need two turns to do that. Likewise, I consider if the tool would only do one thing, or with small extra effort do two, e.g. help fix some raw data files, and also create artificial test data.
  • Future usefulness. It might save me time for this weeks' work, but is it likely to save me or someone time on a new project next week?
  • It is a good time to learn a new language, library, design technique or whatever, and I have the time to do it. The tool is a beneficial side effect of doing something education, using time already granted for education.
  • When we're having serious problems getting things to work. Like skydiving without a parachute, you really should take the time to make or buy a parachute. If you can't get any meaningful experimental results, or the new web app won't work at all, you just gotta stop and make or buy a tool to measure what you can't see, drive components, or replace part of the system. When the work is totally hung up in need of a tool, I don't care about future use or multiple use. Necessity weighs enough.
1
  • 3
    "The tool has more than one purpose" Unless they are actually the same purpose applied in two different ways, in my experience its better to just make two tools. Jul 13, 2013 at 14:05
5

My rule of thumb is that when I have to do something for the third time, it's time to either write a little script to automate it, or rethink my approach.

I'm not making a full-blown "tool" at this point, just a little script (usually bash or python; perl would work too, or even PHP) that automates what I did manually before. It's basically an application of the DRY principle (or the Single Source Of Truth principle, which is in essence the same thing) - if you have to change two source files in tandem, there has to be some common truth that they share, and that truth has to be factored out and stored in one central place. It's great if you can solve this internally by refactoring, but sometimes, this isn't feasible, and that's where the custom scripts come in.

Then later, the script may or may not evolve into a full-blown tool, but I usually start out with a very specific script with lots of things hard-coded into it.

I hate grunt work with a passion, but I also strongly believe that it is a sign of bad or incorrect design. Being lazy is an important quality in a programmer, and it better be the kind where you go through great lengths to avoid repetitive work.

Sure, sometimes the balance is negative - you spend three hours refactoring your code or writing a script to save you one hour of repetitive work; but usually, the balance is positive, more so if you consider costs that aren't directly apparent: human failure (humans are really bad at repetitive work), smaller codebase, better maintainability due to reduced redundancy, better self-documentation, faster future development, cleaner code. So even if the balance appears negative right now, the codebase will grow further, and that tool you wrote to generate web forms for three data objects will still work when you have thirty data objects. In my experience, the balance is usually estimated in favor of grunt work, probably because repetitive tasks are easier to estimate and thus under-estimated, while refactoring, automating and abstracting are perceived as less predictable and more dangerous, and thus over-estimated. It usually turns out that automating isn't that hard after all.

And then there's the risk of doing it too late: it's easy to refactor three brand new data object classes into shape and write a script that generates web forms for them, and once you've done that, it's easy to add 27 more classes that also work with your script. But it's close to impossible to write that script when you have reached a point where there are 30 data object classes, each with hand-written web forms, and without any consistency between them (a.k.a. "organic growth"). Maintaining those 30 classes with their forms is a nightmare of repetitive coding and semi-manual search-replace, changing common aspects takes thirty times as long as it should, but writing a script to solve the problem, which would have been a lunch break no-brainer when the project started out is now a frightful two-week project with the dreading prospect of a month-long aftermath consisting of fixing bugs, educating users, and possibly even giving up and reverting to the old codebase. Ironically, writing the 30-class mess took way longer than the clean solution would have, because you could have been riding the convenient script all that time. In my experience, automating repetitive work away too late is one of the major problems in long-running large software projects.

4

I just remembered this:

xkcd - Is It Worth the Time?

The problem with this of course is that in real life situation you can't easily measure those data to select the right cell in the table. And as it was mentioned in other answers, there are other variables (the risk of error, task is too boring to do it even once, ...) you need to add to the equation.

So my answer is that it really depends on the situation and you can not hope to get any "correct" answer for all the situations. After all life would be boring if we had cookbooks for everything.

1
  • 1
    Nice one! :-) But the time saved might also be on other tasks - either since the tool can be used for more than one purpose, or because of the knowledge you got when writing the tool. Jul 16, 2013 at 12:46
3

This is a big problem in my experience. Tool building is usually left to a motivated developer who stops work to build the tool. This often interferes with the development even if it provides value. Tool building needs to be viewed as an integrated part of the development "Process".

I remember attending code reviews where header errors would result in scheduling another review. Many of these could have been detected by a tool. e.g. incorrect sloc counts, missing requirement, formatting errors. My tool written in perl generated the header from the delivered code and validated requirements from an Oracle database. It was not part of our "Process" so in the short run building the tool was viewed as delaying delivery.

The entire team needs to periodically stop and see where there are manual efforts than can be automated by tool creation.

2

All of the other answers are good, but I'd add one more reason why it's valuable to spend time to build small tools (and customise your .vimrc, .emacs etc.):

Sometimes you get creatively or motivationally "stuck in a rut" and doing something, anything, will "get the juices flowing" again (to mix metaphors). Ideally that would be something that productively pushes the project forward, but if it's a bit tangential and that inspires you, then that's good too.

You might not be staring at a blank screen, but instead simply having trouble seeing the progress you're making on a big task. In that situation ticking off something tangible can stop you feeling like you're not getting anywhere.

A variation on this is when you keep thinking about something that should be on the "back burner" - just taking a moment and doing it will get it off your mind and free you up to put your full energy into the main task again.

1

It depends on what you consider as making a tool. Arguably I make loads of them in ways that other devs would consider frivolous... Because I make them for just about everything except the most basic filesystem commands.

I use 2 rationales to support that.

  • It's an extension of the DRY principle. If we want to repeat something, manual effort is the least effective use of human resources.
  • It's an effective way to record what I did, so if I built something then I (or someone else) may want to refer back to how I did it weeks or months later and it keeps the log of work with the project.

Occasionally they grow into bigger tools and gain interactive functions, but if they don't they still have value as a record.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.