I'm no software engineer. I'm a phd student in the field of geoscience.
Almost two years ago I started programming a scientific software. I never used continuous integration (CI), mainly because at first I didn't know it exists and I was the only person working on this software.
Now since the base of the software is running other people start to get interested in it and want to contribute to the software. The plan is that other persons at other universities are implementing additions to the core software. (I'm scared they could introduce bugs). Additionally, the software got quite complex and became harder and harder to test and I also plan to continue working on it.
Because of this two reasons, I'm now more and more thinking about using CI. Since I never had a software engineer education and nobody around me has ever heard about CI (we are scientists, no programmers) I find it hard to get started for my project.
I have a couple of questions where I would like to get some advice:
First of all a short explanation of how the software works:
The software is controlled by one .xml file containing all required settings. You start the software by simply passing the path to the .xml file as an input argument and it runs and creates a couple of files with the results. One single run can take ~ 30 seconds.
It is a scientific software. Almost all of the functions have multiple input parameters, whose types are mostly classes which are quite complex. I have multiple .txt files with big catalogs which are used to create instances of these classes.
Now let's come to my questions:
unit tests, integration tests, end-to-end tests?: My software is now around 30.000 lines of code with hundreds of functions and ~80 classes. It feels kind of strange to me to start writing unit tests for hundreds of functions which are already implemented. So I thought about simply creating some test cases. Prepare 10-20 different .xml files and let the software run. I guess this is what is called end-to-end tests? I often read that you should not do this, but maybe it is ok as a start if you already have a working software? Or is it simply a dumb idea to try to add CI to an already working software.
How do you write unit tests if the function parameters are difficult to create? assume I have a function
double fun(vector<Class_A> a, vector<Class_B>)and usually, I would need to first read in multiple text files to create objects of type
Class_B. I thought about creating some dummy functions like
Class_A create_dummy_object()without reading in the text files. I also thought about implementing some kind of serialization. (I do not plan to test the creation of the class objects since they only depend on multiple text files)
How to write tests if results are highly variable? My software makes use of big monte-carlo simulations and works iteratively. Usually, you have ~1000 iterations and at every iteration, you are creating ~500-20.000 instances of objects based on monte-carlo simulations. If only one result of one iteration is a bit different the whole upcoming iterations are completely different. How do you deal with this situation? I guess this a big point against end-to-end tests, since the end result is highly variable?
Any other advice with CI is highly appreciated.