Storing large binary data for integration tests

Question

I have the following requirement for an application:

• Store large files (order of megabytes) which I will use as input for non regression test and some expected properties of the processing result
• Allow business users to add additional files and properties
• Being able to track over time which files (and properties) were used for testing

Obviously putting the test data under VCS wouldn't allow my business users to add files and properties easily, should I use a cloud storage? What other solutions do I have?

Answer 1

I would argue that you want this in VCS.

When you branch or tag the code, you're going to want to branch/tag your tests too. And with that your test data. Otherwise how do you know which data set ties up with which particular code set that you have.

You could possibly contain a reference to a dataset within your versioned codebase, and store the actual dataset outside. Then your tests would have a referenced dataset to load and you could change that with each branch or tag. But that seems particularly troublesome and difficult to maintain, and I would avoid that if at all possible.

Answer 2

Obviously putting the test data under VCS wouldn't work

Why not? And what about LFS?

Even without LFS, you need to think twice before discarding version control solution. You tell only that there are some files which have a size of a few megabytes. No further indication.

• A dozen 5 MB files which change once per month for a 12-months project is not a big deal: any version control will handle that painlessly.

• Hundreds of 5 MB files changed nearly daily for a 2-year project could bring some version control systems/servers/networks down and require additional preparation.

If you're in the first case, simply create a dedicated repository using your preferred version control system.

If you're in the second case, then you need to be very specific: how much files? How frequently are they changed? What is the exact average size? How long is the project? From there, you may either use an existent solution, such as LFS, or create your own, where files will actually be stored on a NAS, but never accessed directly, but only through the custom version control.

If you're in Microsoft world, FILESTREAM feature of Microsoft SQL Server could help abstracting away the actual storage of the files. Eventually (although I don't know Microsoft SQL Server well enough), you might be able to combine it with CDC to track the actual changes. Also, SharePoint provides you with the capability of versioning binary files while storing them efficiently if only part of a binary file is actually changed.