I have smaller unit tests that use small snippets from real data sets. I would also like to test my program against full data sets for a multitude of reasons. The only problem is that a single real dataset is about ~5GB. I haven't found any hard numbers for what Git repositories can store but that seems like too much.
The solution that my team has adopted is that the project has a file that contains a path to a network attached file system that holds our test data. The file is git ignored.
I feel like this is an imperfect solution for two reasons. When the NAS isn't working, is slow, or is down than we can't run a full test. The second reason is that when someone first clones a repository the unit tests fail so they have to figure out how to mount things with a certain name and the syntax used to build the testing path file.
So my question is two fold. How much data is too much data to store in revision control?
What is a better way to handle large amounts of test data?