What is the right way to manage developer scripts?

Question

Developers create scripts to help in their work. For example, to run Maven with certain parameters, to kill unneeded background tasks that crop up in development, or to connect to a certain server. The scripts are not core build scripts nor are they used in our Continuous Integration server.

What is the best way to manage them? To put them into a directory (maybe /scripts ) and check them into Git? To maintain them separately in some file server?

The argument for treating them as source code is that they are source and can change. The argument for not doing it is that they are just auxiliary tools and that not all developers need any given script (e.g. Linux-specific scripts where some developers work on Windows).

Answer 1

Developer scripts go also into version control, because usually these scripts also depend on the items in version control, e.g. file paths.

If these scripts are versioned they also should work for all developers to avoid that every developer writes his own set of scripts, which becomes a maintenance hell.

In addition bugfixes or improvements of these scripts are automatically rolled out to every developer via version control.

Answer 2

In addition to @simon's answer.

Not all in the software engineering is about programming, designing or modelling. There's a myriad of tasks we perform continuously during the working day. You already mentioned one -building the project outside the IDE- but there are many more.

Experienced / proactive developers tend to automate these tasks. Some, even build tools when these tasks become part of the SDLC and they are tedious -and prone to error- to do by hand. Programs are good at doing repetitive jobs, no matter how tedious they are. We -humans- are not that good.

These tools/scripts have others positive side-effects

1. Productivity
2. Transfer of knowledge
3. Autonomy (for newcomers)

So, yes the scripts should be in the SCM and they should be one more tool in the developer's toolbox.

Regarding the folder /scripts I would say that It doesn't matter. For simplicity I leave them in the project's root directory so that all the routes declared in the scripts are relative to the project's folder. If I need access to external folders or files, I create soft links.

Things to consider before checking the scripts into the SCM.

• For security, make sure the scripts have no hardcoded credentials -ideally, the scripts should be well parametrized-

• Make sure the scripts don't do odd things to the system, as for instance to execute commands that can not be undone (the most typical rm -rf).

• Since these become part of the project's source, documentation is highly appreciated.

• Scripting is not rocket science. Make scripts concise. Instead of one to rule them all ... and in the darkness bind them, make more, smaller and concise. As if you were applying SRP.

Answer 3

I'm going to offer a somewhat more negative opinion. On the one hand, developer scripts that are generic, effective, and useful should of course be shared with other developers, and the best way to do that is to have them sit with the code in the same repository.

However I would set a high bar to entry for having scripts be committed. Scripts are code, just like the software itself. That means they need to be treated similarly to other pieces of code:

• Going through code review
• Tested and automated if possible
• Considered when making changes to the codebase (specifically, if a script is used by many developers, making a change that breaks the script is going to cause a lot of strife)
• Maintained (with everything that entails - prioritization, time, documentation etc.).

There are a number of further considerations that apply more to scripts than to the software itself:

• First and foremost, it is much more difficult to convince one's organization and stakeholder to invest in maintaining scripts that make developers lives easier. That means it's harder to get the time to meet the above criteria - it's easy to write a script that works for you on your environment, but parameterizing it, making it stable, documenting it takes much more time. This means that scripts can and will become dead code unless a developer can justify keeping the script current.
• It is much less likely that multiple developers will be familiar enough with a complicated script to maintain it, or that multiple developers feel ownership of the code. When the original developer leaves, finding someone else to take ownership from can be difficult (and finding and justifying the time for them to learn how the script works can be even harder).
• It's much more likely that a script will interact with the developers machine and build environment in some way. It's also very likely that you will have multiple developers with multiple different environments. If a script messes up an environment because it wasn't maintained properly or a corner case wasn't considered, you are not just breaking a nightly build of your software, you are potentially costing a developer a day or more of work to get their environment back to normal. This can cause baad blood and a loss of trust.
• Because scripts are often external to the software itself, maintaining them can be a challenge. If the scripts are not run though automations, it is easy for them to go stale or to be forgotten, at which point they have become dead code and are just tech debt that someone will need to take time to clean up.

To summarize, scripts can be very helpful for an individual developer, but sharing then as part of the codebase itself can be a much more difficult task, and can potentially cause more problems than are solved.