How to open-source a project whose git repository has copyrighted media in the history?

Question

I want to release an audio fingerprinting software project under a free license, but the repository contains copyrighted audio files. The test cases also currently use these files. How do I release the code to the public with maximum version history but without violating copyright?

Details:

• The code is versioned under git. We will collapse it all back into one branch before release.
• There are 400 MB of audio data. Some files are free-licensed music from e.g. Jamendo, others are MP3s from our personal collections.
• No matter what approach we take, we'll always keep an immutable copy of the original repo, so as not to destroy project history.

Main question: How to handle the public release?

1. Expunge all history of the files in question from the git repository and release the altered repo. (v64 pointed out a way to do this.)
2. Alternatively, take a snapshot of the current state of the code and don't even bother having a public history of the pre-release code.

Side question: How could we have avoided this dilemma in the first place, given that sometimes private code or media is needed for the early stages of a project?

Answer 1

GitHub has a page explaining how to expunge a file from all history: Remove sensitive data.

From time to time users accidentally commit data like passwords or keys into a git repository. While you can use git rm to remove the file, it will still be in the repository's history. Fortunately, git makes it fairly simple to remove the file from the entire repository history.

Danger: Once the commit has been pushed you should consider the data to be compromised. If you committed a password, change it! If you committed a key, generate a new one.

Purge the file from your repository

Now that the password is changed, you want to remove the file from history and add it to the .gitignore to ensure it is not accidentally re-committed. For our examples, we're going to remove Rakefile from the GitHub gem repository...

Answer 2

Side question: How could we have avoided this dilemma in the first place, given that sometimes private code or media is needed for the early stages of a project?

If you're going to track large media files (400MB of audio), put it in a separate repository.

That kills two birds with one stone:

1. The main repo is 400MB smaller. (People don't have to download 400MB worth of content each time they clone.)
2. The media can be private and is kept separate from all the other stuff. As such no extra work needs to be done to release the public repository.

If you like, you can make it more convenient to work with by making the media repository a submodule of the public repo (that you plan on releasing).

That way you just keep a pointer to it, not the (sensitive) content itself (for early stages of development). Then when you're going to release the repo publicly, just remove the submodule reference, which is a lot less troublesome than rewriting your history to filter out 400MB worth of stuff.