How should "useful throwaway" scripts be handled?

Question

You know how it goes: there is some small repetitive task for which you found a way to quickly automate 95% of the work. You create a script, run it, manually fix the output, and you're done. Of course you don't commit the script, as it doesn't match the company quality requirements (it doesn't have documentation, nor does it have any tests).

Some time later you see a colleague working on a similar task. You go "Hey! I made a script for that. Let me look that up. [looks] Oh, it was stored on my previous laptop so I don't have it anymore. Too bad."

These scripts can often save lots of time, and as the leader of the team, I would like them to be stored in version control. However, if I impose the same rigorous standards as the rest of the code base to these scripts, I'm afraid most developers will keep them to themselves.

The only other option that I can come up with is to let developers store the scripts in a special part of version control, in which there is no quality control (much like GitHub Gists). The risk is that others will not be able to use the code because they cannot find or understand it.

How could this problem be solved? Or should it not be solved?

Answer 1

The only other option that I can come up with is to let developers store the scripts in a special part of version control, in which there is no quality control (much like GitHub Gists). The risk is that others will not be able to use the code because they cannot find or understand it.

How could this problem be solved? Or should it not be solved?

The problem cannot be solved.

There is a big difference between sharing code by word-of-mouth when you overhear that a colleague is facing a particular difficulty, and sharing it simply by putting it into a library to which all have access.

The difference is that, in the first case, you are playing the role of the creator and librarian with implicit knowledge of the problem and the code you have which solves it, whereas in the second case, those roles are not present.

Your general standards of code may be designed to eliminate the role of the librarian in your company, and ensure that the library remains self-navigable to all-comers.

With a huge edifice of long-lived code (as characterises most core software products), time and effort is justified to preserve the ability of multiple generations of workers to continue working with the whole edifice. In companies with a high turnover of staff, a "generation" of staff may be as little as every year or two.

But of course, attaining those "librarianless" standards comes at the price of huge time and effort for those creating the library in the first place, and it still demands significant time and effort from those who later come to use and navigate the library (though not the same effort as re-creating the entire edifice from scratch, which may have already taken multiple generations).

On the other hand, most of these quick-and-dirty scripts will be written to get simple things done quickly - perhaps on a one-off basis. They're not intricate or robust creations.

You say they "save time", but of course this only remains true when they are not written to the much higher, librarianless, standard. They are the personal construction jigs of those who made them - the temporary levers and scaffolding of human effort, not the final durable product.

It should be possible to check them into version control, so that the creator has somewhere to systematically store and preserve them - but it should also be clear that they represent product over which their creator still has oversight.

If you can read their code and immediately see what it does, so much the better. But it is the creator's personal library, and they are not being held at the standard that other people must be able to see what it does without reference to the creator.

This is not a "risk" borne by the team, any more so than the arrangement of personal effects in your house is a "risk" to those who may wish to ask to borrow them. On the contrary, having to index the entire contents of your house, arrange them in a set fashion and provide an instruction manual for every item, is a serious risk to your own ongoing efficient use of the house.

Answer 2

I feel your pain. I'd say having a separate part of your repo is probably the option for storing these scripts. However, this only solves one part of the problem. You now have them stored somewhere, but no good way for your colleagues to know they are there.

In my experience as a consultant for a company where scripts are abundant and written for different customers and usually on site, finding out what is already there is hard!

You should find a way to share what you've built.

In our company, as we are not that big, we check things like scripts into our repo, but also send out a company-wide email to inform our colleagues of a new solution. This way, at least everyone knows what is already out there and who to contact should they need it.

Answer 3

Others have already explained how and why to ensure that your 'useful throwaway' scripts can be made good enough to go in the repo. I'm just going to butt in and say that it is just as important to ensure that, if they are not good enough to go in the repo they must get thrown away.

To put it another way, they are either good enough to go in the repo or they are not and must be thrown out. If they are not thrown out they become an unmanaged, and eventually unmanageable, maintenance burden.

Answer 4

I would first start by saying that if your developers are reluctant to contribute code because of your standards, either your standards are too rigorous or you should improve the quality and work ethic of your developers. If these are small scripts, there shouldn't be much extra effort required to put a couple comments in. Developers should be writing readable code by default, and if they aren't, your real problem lies in your staffing.

To answer your question, you have a few options:

1. If the scripts are generic enough, you could host them in a separate repository. If the scripts are very project-centric, this may not work.
2. Host the scripts in the same repository in a directory of their own. For example, at my company our projects usually have a cli/ directory which contains a handful of command line scripts, usually meant to be run one-off to initialize environments, import certain pieces of data, and so on.
3. This could be a poor option depending on what your needs are, but if you have some reason you can't commit code that doesn't expressly follow your standards, and/or you have developers who aren't willing to write scripts if they have to follow those standards, you could host the files on a shared drive or something similar. This may not be the greatest solution if you need all the benefits of source control -- however it can be beneficial in some circumstances (thanks Doc Brown for this edit)

With either of the first two options, you can choose to enact more lax coding standards on either the other repository or the directory.

Answer 5

Start formalizing your processes.

You don't give many clues to what these scripts are being used for, but I assume you have various various deployment and maintenance tasks, or common data fixes which are handed off to developers to do because they are the ones with enough knowledge to fix things?

You should try to become more professional about this.

• Write down what these tasks are and the steps to complete them.
• Publish that document on some internal website or intranet
• Record how often the task is done and how long it takes in a ticketing system.
• Automate the manual steps with a program, but don't skip source control, testing, support etc. Develop a proper admin tool.
• Try to use a 3rd party tools rather than developing your own solutions where possible. It will make it easier for new hires

The goal is to make the process purely a rote function that anyone can do by following the document, or running the tool.

Then you can move your developers back onto writing features for your product and hire some support staff to deal with day to day issues

Answer 6

Almost any script can have potential for re-use in one way or another. But that potential won't be apparent initially - which is exactly why it's considered throwaway at the time.

Offer your team the Github Gist-like repositories solution, with no quality control whatsoever, to encourage its use and prevent loss of that potential.

But also have one or more repositories for shared tools/utilities, with appropriate quality control in place.

Only when the scripts stored in that throwaway repository are referenced their potential for re-use starts to unravel. That's when the formalized re-use should kick-in, intiated either by individual developers recognizing that potential or by you if the references occur in a team context. It can happen at the very first reference, or maybe at subsequent ones (i.e. when the re-use potential is confirmed to be higher), depending on the team's ability, schedule, load, etc.

The risk is that others will not be able to use the code because they cannot find or understand it.

The reference itself takes care of the find problem. The author of the script and/or the person making the reference might be able to help with the understand problem. If not - either the re-use potential is not really there or understanding the code would be just part of the formalized re-use effort/cost.

Once intiated, the formalized re-use effort should go on the team's radar screen and the result should eventually land in the shared tools/utilities repository. That's when the equivalent throwaway script(s) from which the re-use originated can be deleted (if their potential was exhausted).

Answer 7

Focusing specifically on badly written but functional code; there is a very strong argument to not put these in a repository.

I have a collection of snippets in my personal Dropbox folder. It contains things I find useful:

• A generic EF repository example
• A simple serializer/deserializer
• A small application that crossjoins two CSV files and outputs a new CSV file.

However:

• The EF example lacks a unit of work implementation and contains nothing but the most rudimentary CRUD options.
• The serializer is based on the old XmlSerializer and it's nigh impossible to swap it out for anything else. It also has a massive circular reference issue.
• The application is hardcoded to look for a.csv and b.csv in the application directory, and outputs a hardcoded ab.csv in the same directory. There are no checks if files exist, there is no nullreference handling.

While I still use these snippets regularly when I need to quickly implement something, they are in no way reusable or well-built. There are many reasons why this can't be put into our company's version control:

• Not self-documenting.
• Little to no coding style other than my own hackstyle.
• No error handling.
• Not all pitfalls are obvious (e.g. the circular references in the serializer)

But the most commonly shared trait of all these snippets:

• Even if I were to create documentation, it'd probably take you longer to read and understand it than it would to make something yourself!

These snippets are good for me to use, because I personally remember what they were used for, how to use them, and any pitfalls I encountered.
But if I were to give someone else access to these snippets, without me being present, then they're not going to think that these snippets are in any way helpful.

The snippets are only an extension of my experience as a developer. Without me, these snippets are garbage. I only use the snippets because I'm not able to remember an entire class' syntax down to the last character. But I do remember the intention and pitfalls, even long after having last used the snippet.

---

Think of it this way:

Duct tape is incredibly versatile, it can solve many problems. However, things that have been solved by using duct tape are generally considered as unsightly. If IKEA were to include duct tape in all of their flat packs, while it may actually help some people who need it, it would also make a strong statement about IKEA's trust in the quality of the furniture they have sold you.

For the same reason, companies shouldn't allow hacky snippets in their version control, as it calls the quality of the overall product into question.