1

The Issue

When I'm fixing a bug during web development projects, I often find myself cleaning out the existing (tainted) database records, clearing out log files, etc.

I do this so that I can start from a fresh state to ensure that none of the previous buggy run's records will affect my new run.

Dropping tables, clearing log files and caches, etc. get pretty tedious to do everytime the server is restarted, so right now I have a simple shell script that I run called drop.sh that automates that process. It looks something like:

#!/bin/bash
mongo <dbname> --eval "db.dropDatabase()"
cd /project/directory
rm ./logfile1.log
touch logfile1.log
rm ./logfile2.log
touch logfile2.log

But I feel this is very clunky and also not sure whether to include this script in the .gitignore.

My Question

I'm a new developer who has no real world experience yet (student). I'm guessing that this isn't what's actually done in industry, and wanted to ask what the best practice is to automate something like this.

Should I be using a Makefile instead that can clear out the server state?

Should I be using automated build software like Jenkins or something?


In case it matters, the two projects I'm working on right now that deals with this issue is:
  1. A python script that works with MongoDB database
  2. A Node.js + Express server + PostgreSQL database for a full-stack webapp
3
  • I would suggest to look into docker. – JayZ Oct 15 '20 at 12:09
  • 2
    Maybe not a direct answer to your question, but a relevant consideration: your situation is exactly why I choose to use in-memory databases when I'm debugging. This means that whenever I start a debug session, the database begins from an empty state (or non-empty predefined default state, if you set up an automatic seed). This does require you to abstract your code so that you can change your data source (real database vs in-memory database), which may be very hard for a beginner. But this is something you will have to learn as a developer eventually, so it's good to already keep it in mind. – Flater Oct 15 '20 at 13:09
  • 1
    Flater's solution is good and it works great with change loggers, frameworks which allow you to load data during the bootstrap so that you can put the database in one or another state. This is very very useful for testing and compatible with local development. However, it depends on the stack you work with. For example, you could do this in Java either for that MongoDB and the PostgreSQL, but don't know if Python has anything similar. – Laiv Oct 16 '20 at 6:50
2

Dropping tables, clearing log files and caches, etc. get pretty tedious to do every time the server is restarted, so right now I have a simple shell script that I run called drop.sh

Any task you find to be tedious or repetitive, it's likely to be so for whoever inherits the project or whoever is working alongside with you.

Automating repetitive task by implementing your own tools is (implicitly) what you are paid for. You are not only professional by doing this, you are a good one because these tools make you productive.

But I feel this is very clunky and also not sure whether to include this script in the .gitignore.

Any script should be in the right folder within the project and in the SCM. Absolutely. Because you will make others as productive as you by sharing these tools. Moreover, others might contribute to the improvement of these tools, making the day even more productive.

But it's not only about to be productive. It's also about knowledge/code reuse. These shells (if written properly) can be re-used. As you said, you can reuse them for CI.

It's only "clunky" when it's not well implemented or when it's not well documented. Introduce all these shells in the README.md file!

Should I be using a Makefile instead that can clear out the server state?

You should perhaps implement both! If your project is the one of those that Makefile allows 3rd parties to compile and assemble quickly, then go ahead. If you Makefile solves your compiling, assembling or packaging then go ahead. But don't mix them up. Don't make a God Shell/Makefile.

Resetting dev environment is one job, compiling and packaging is another job. Releasing the binary is a different job and rolling out the binary is... a different job too. For simplicity make 1 tool for each so you can orchestrate them in the order that best serves you. It's also easier to reason about if you have a tool that does only one thing.

Back again to the beginning, documentation is a must. Make sure you document well-enough these tools.


1: This is the main reason why I programme these shells, why I make them part of the project and why they are in the SCM. Because this way, I can tell Jenkins _- do use this shell to do this task - _ rather than rewriting the whole task in a different language. Again. It's likely whatever works in my local env, will work in Jenkins (master or slave doesn't matter). If I work on Windows, I implement the respective .cmd shells.

1

Your approach looks reasonable. Automating repeated tasks is absolutely good practice, and clearing out development databases helps create a reproducible environment. Doing it with a makefile instead of a shell script wouldn't really be better since there is no network of dependencies.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.