My research team and I are using Git to manage our various scripts and notes/publications for our on-going projects, and I'm wondering about what is considered best practice for the situation we are in.
Until now, the scripts we were using were fairly small and did not require too much computing power. We were usually agreeing on what a script should do; one or several people coded it, ran it on their personal machine, and pushed some output file on the master branch. We did not follow a specific workflow and were mostly using the repo like a Dropbox: since we are a small team, someone pushes to the master branch once they are finished with whatever they are doing so everyone else can see it.
Recently we have been needing more computing power and got access to a high performance machine where we can run our scripts for several days. Our workflow is currently the following: we code the scripts and test them on our personal machines for small runs; once we are ready to do some extensive run we put all the necessary files into a
production/ directory, and I
scp it to the server.
We cannot clone the whole repo on the server because we have a limited amount of space we can use, and we are not comfortable copying unpublished results and other internal files on a machine we share with other people.
We've considered creating another repo just for that, but it does not seem better than what we currently do as in the end I would still need to copy the files from the main repo to the new one. I have also read about sparse checkouts, but as far as I understand the whole repo is still downloaded.
Is there a best practice or standard workflow to do that kind of things?