How to Maintain Rarely Used Scripts

Question

I come across the need to do a lot of one time scripts (related to API evaluations, data extraction, experiments etc.) that have the potential only to be used very rarely in the future. These could be one function python scripts, jupyter notebooks etc. Although they are rarely used, they are useful if the need to use them arise in the future. I am finding it very difficult to manage and maintain such scripts. They might not necessarily fit into any git repo in our company (the company mostly maintain source code related to products on git repos). How would one deal with such script in a systematic manner? Ideally I would like to keep such scripts in one repo dedicated for such scripts and have the scripts divided (grouped as sub-folders) into the job they belong to. However, such a repo would represent only my scripts and I don't know whether that the right way to do it. Currently, these scripts are scattered on my local computer and some on certain servers I normally work in.

Update:

The following two questions don't address what this question is referring to

Best practices for sharing tiny snippets of code across projects

What is the right way to manage developer scripts?

My question is regarding scripts that were important one time for the company to perform some operation (like an API evaluation) and would very rarely be useful in the future. But these scripts are important enough to be held onto because there could be instances where you would need to reuse part of such scripts to do some similar job in the future.

Answer 1

There is no single right way to do this. Bear in mind a few best practices for code in general:

• Use version control
• Put it some place you can access
• Organize stuff in the repository
• Document things where necessary

Pretty generic. Your idea of using folders to categorize things is a good idea. Put comments at the beginning of the scripts to summarize why it was created, and what problem it solves. Use a well known comment-based documentation format for whichever programming language the scripts are written in. Don't be afraid to come up with your own way of "tagging" scripts, since they likely will fall into multiple categories of things that a folder hierarchy cannot replicate.

And most importantly, don't over think this. Put it in version control, and make sure the version control is properly backed up and on organization-supported hardware. Add comments in the scripts to ensure you can come back 6 months or one year later and know what the script is used for.

Answer 2

Beware that even for the 5-man-day scripts you mention, if they were written off-the-cuff (i.e. unsupported by the formalities of strategic, team-oriented IT development) then after six months or a year you may not remember all the subtleties they incorporate and whether they are suitable for reuse.

My attitude to such scripts and similar kinds of tactical development of my own, is either to accept the need to organise them if they are strategically useful, or else to throw them out after maybe a month or two of disuse.

In terms of how to organise, if there is no existing structure into which things fit, then I tend to just prefix a sequential number to the folders/filenames. This permanently maintains the relative order in which they are produced, which tends to be (a) something a person remembers by default, (b) supports a search strategy which seems to remain viable even after selective deletions, and (c) emphasises their age during a periodic review of their retention.

Sometimes, it becomes obvious that you have accumulated several similar things, which can then either be managed and co-located as a set, or merged into fewer more general artefacts.

It's sometimes hard work to sift and thin down a collection of such scripts if you write a lot of them, but endless accumulation is always the very worst choice, because even if value exists, an unmanaged collection will make that value increasingly impossible to find and discern as the collection grows, and the sheer weight of artefacts will impose a mental burden just by their existence.

As for sharing scripts with others, there is virtually no prospect of others using anything which isn't well-documented, easily accessible, and substantial enough to be worth searching for, unless you perform the role of librarian and provide the solution to them.