I'm thinking about how to decide whether it's better to encapsulate my work behind well-named function names, or to expose it - which will help developers understand what's going on more quickly? Is there a name for the study of this sort of problem?

Specifically, if I'm running a bunch of bash commands ultimately, but I have significantly complex logic around those commands, at what point does it make sense to write this in a high-level language like Python, even though this obfuscates the actual bash commands being run?

Detailed problem

Currently I'm trying to write a Jenkins build script for my project with roughly the following steps:

• Pull my code from github
• Compile sass files into CSS
• Pull down a sub-folder from a different github project
• Zip up the project
• Upload it to an object store with a unique ID

I'm thinking about how to write this to be as easy for future developers as possible (this code is never going to be seen by end users). These developers are likely, but not definitely, going to be fairly good at Python. They will definitely have a passing familiarity with the command-line, but are likely to be unfamiliar with more complex bash scripting.

The first iteration of this build script was just a list of sequential commands, something like:

git clone [email protected]:username/project.git
git clone [email protected]:username/sub-project.git project/sub-project
sass --update project/css
tar -czf project.tgz project
swift upload my-container project.tgz --object-name=project-`sha1sum project.tgz`.tgz

However, this set of commands quickly became more complex as I started to do things like only clone the git project if it wasn't already there, otherwise update it - to speed up the build. Before I knew it I had 50 lines and a fair few conditionals.

So the first thing I did was encapsulate these into bash functions, e.g. update_git_dir, so my build script looks more like this:

#!/usr/bin/env bash

source helper_functions.sh

update_git_dir project [email protected]:username/project.git
build_sass project/css
create_archive project project.tgz
upload_to_swift project.tgz

This is one level of encapsulation. Now the developer, who would have understood the git clone etc. commands directly, can't actually see what's going on. They have to look in helper_functions.sh.

However, as time went on I realised that many of my helper functions now consisted of more conditional statements, variable assignments and function calls than actual commands. These conditional statements can be quite opaque to someone not familiar with bash scripting:

# Get revision ids
dependencies_requirements_revno=$(cat ${project_name}/pip-cache/requirements-revno.txt)

requirements_context=${project_name}/${requirements_file}
requirements_dir=$(dirname ${requirements_context})
if [ "${requirements_dir}" != "${project_name}" ]; then
    requirements_context=${requirements_dir}
fi
latest_requirements_revno=$(bzr-revno ${requirements_context})
latest_revision=$(bzr-revision-id ${project_name})

So I started migrating my code into Python. So now my build script looks like this:

#!/usr/bin/env python

from builders import GitProjectBuilder

builder = GitProjectBuilder(
    project_name='my-project',
    swift_container='my-container',
    git_repository='[email protected]:username/project.git',
    sub_project='[email protected]:username/sub-project.git'
)

# Compress and upload
builder.build_sass(directory='css')
builder.get_sub_project(repo='[email protected]:username/sub-project.git')
builder.build_archive(name='archive.tgz')
upload_location = builder.upload_archive_to_swift(archive='archive.tgz')
print upload_location

Now, when you look in builders.py, it's much easier to understand the logic - if statements and function calls are much more readable - but now we're even further away from the real shell commands. In my python code the closest I get to directly running shell commands looks like this:

def build_archive(self, archive):
    print subprocess.check_output(
        (
            'tar --exclude-vcs --create --file '
            '{archive_filename}.tar {project_dir}'
        ).format(
            archive_filename=archive_filename,
            project_dir=self.project_name
        ).split()
    )

If the developer needs to work out exactly which commands are being run, it's now much more difficult.

Wrap up

So how do I decide which is the best architecture to maximise transparency while encapsulating complexity?

This problem seems similar to when I'm working with dependency injection where the more dependencies I inject rather than encapsulate, the more complex my initialisation code gets - and I have a similar problem drawing the line.

Is there a name for this field of study?