we have a legacy project, on Windows. It goes on from 1990-s.
Until very recently it was not backed by any version control, today it is moved into Git.
The question is all the prior snapshots. As of now, they are spread on network shared folders, with folder names usually suggesting the part of the project, the date or product version, and often the specific programmer, from whose development machine it was copied.
Needless ot say, it takes much of disk space, and mostly on many copies of exactly same files, and today when Covid-19 makes many of us work from home - that is becoming a problem.
Those legacy sources are rarely needed, but sometimes one just to look for a specific identifier (variable, function, file, etc) or even a statement - to find when and why it was introduced.
Copying them onto my home machine would be slow and would occupy too much of my local disk. Same for other developers. Even if i would Zip (or PPMd) files on server - it would be still big blob to copy over RDP, and i would not be able to search for the text in that archive without speed impact, or without unpacking it first - which will take a huge hit on local disk space on every developer machine.
I can not move them into existing Git repository - when i settled for my own convenicenceit i did not even knew about those archives, less so had access to them.
Anyway, Git is not a solution here. Git (or other DVCS) will do data de-duplication and will greatly decrease disk space required, indeed. But Git mantains a "one true current snapshot" model, which is useful for development, but not for "industrial archeology" (yes, i can settle several repositories and switch them to different branches - but that defeats the idea). I just can not run the text search through ALL the snapshots stored inside Git repo, within all their branches and revisions/commits, with the same ease i can run
grep (or a similar GUI tool) over thousands of mostly plain text files, even on a shared network folder.
Additionally, i do not have information to recover the "real" scheme of all the "branches" that were happenning with people i never met. While guessing timestamps of every folder is doable even if time consuming, inventing some formally correct relationship of snapshots like "from John" and "from Mary" is not.
Additionally, putting files into Git will loose their timestamps metadata. While it is not needed much for the sources that were in DVCS from get go, for pre-VCS sources that has a lot of meaning. Even for mere guessing which modules were alive and which were derelicts that people just could not practically delete away without VCS.
So, how people do manage situations like this?
I think i need some system that:
- runs on Windows
- deduplicates files
- maintains original paths, names and timestamps
- has useful and easy to learn FTS without restricting me to one "current" snapshot out of many stored
- preferably FTS to be available as usual "Search text in files" with no practical performance hit when compared to just storing files and folders "as is" (so, Zip archive is not an option, and it has no deduplicaiton too), so zero learning curve and freedom to use any grep-like and diff-like tools of choice one might like.
- does not require formal relationships from one snapshot to another, though optional timeline relations would be good bonus. It would be okay to have no concept of snapshots at all - it would be no worse than what it is today.
- can be easily and efficiently copied over the relatively slow network (so, if there is some natively de-duplicating filesystem for Windows Servers - it would not help much, it would be problematic to copy the data over network without re-duplication and there will probably be no such filesystem on non-Server development machines)
- low maintenance burden
- preferably detects and fixes or at least warns about random data degradation events (bit flips on HDD/SSD, etc).
- no need to modify data after initial storing it, actually it better be strictly read-only
I think hypothetically it can mostly be implemented on top of NTFS 5 symbolic links, and perhaps someone had already done it?
Or maybe i just fail to think out of the box and see a different workflow, easy to learn and use.