Self-contained projects: Gradle & Maven dependencies offline

Question

I'm trying to get a Java programming project set up in a way that it can be built completely offline just from the files in version control, on any machine, with no prerequisites to install. (Sole exception: the JDK)

I don't know much of Maven, but I'm learning Gradle currently. For the build I'm using Gradle with the wrapper, I modified the gradle-wrapper.properties so the distributionUrl doesn't point to a remote URL but instead a file within the project's folder. So Gradle itself is done already.

For dependencies though, this is a pure nightmare. I've already tried managing it with simply directly pointing to single JARs or collections of JARs, but as you sure know this ends up in total confusion if you have interdependencies and/or transitive dependencies.

Basically what I want to have is Gradle/Maven configured (just for this project) in such a way that they use a different directory for their local dependency cache - not one that is shared by anything you do on your user account, in your home directory, but instead a folder within the project folder (so I can check that into version control as well) which is only used for that one project. I would then only once take Gradle/Maven in online mode, run the build once to download all the currently needed dependencies into the local cache directory within the project folder (whose contents then are put into VC), then put it in offline mode for all subsequent builds.

I haven't found a way yet to reconfigure that local cache path just for this one project (has to be something I can just put into my build.gradle). Anyone have a lead for me?

Edit: Another thing would be to have this local repository within the project folder be a "real" Maven repository (instead of just a cache). But that would involve manual work for one way I imagined: You start a local copy of Sonatype Nexus that has a proxying repository set up for any Maven repos you're referring to in your build.gradle; you modify the repo URLs in your build script to point to your local Nexus, run the build once so Nexus downloads the deps and puts them into its work directory in the "usual" Maven repo structure; then you copy that structure to a folder within your project folder and change the buildscript repo URLs to point to the folder you just copied the files to. Quite a lot of work that can barely be automated.

Answer 1

Alright, so I found a way which works for me, and it's automatable.

I found this snippet which allows you to create a Gradle task that will copy certain project dependencies to an arbitrary folder in Maven directory structure. I needed to tweak it slightly (link broken) so it doesn't just do that for one single configuration, but for any and all configurations the project defines.

My build script looks like this now:

apply plugin: 'java'

repositories {
    maven {
        url './dependencies-maven/'
    }
    mavenCentral()
    jcenter()
    // whatever other external repos your deps come from
}

dependencies {
    // ...
}

task offlineRepo (type:OfflineMavenRepository) {
    repoDir = new File(project.projectDir, 'dependencies-maven')
}

You then simply fire the offlineRepo task once: all current project dependencies will be downloaded to Gradle's local cache and then copied into a subfolder in the project directory (in Maven repo structure). The next time you run the build with the project directory in this state, the just-populated local repo will be queried first, and since it contains (should contain) all the deps, nothing should be taken from the Gradle's user folder cache nor from the web.

At this point you can take the whole project folder to any other machine with just Gradle installed and it should be able to do the build without ever going online to fetch some additional data.

Whenever you want to update some project dependency or add new ones, you'll just need to fire the offlineRepo task once again. Be aware that this won't delete the old version of the dependency you update, so if you don't want to end up with a cluttered repo, you need to do some manual cleaning here.

With this, plus putting the Gradle wrapper offline as well (described in my original question), I've now made my project buildable completely offline and with minimal prerequisites: no internet connectivity required, not even a Gradle install required. Merely the JDK.

Note: When I set this up in a completely empty dummy project, I had to create a java source directory and some sample java file, otherwise Gradle would only download/copy some POMs but not the corresponding JARs.

Addition: The tweaked snippet will only take care of project dependencies. But if you use a custom plugin, which is a buildscript dependency, the script will miss it. You'll need to add more code to the build() method:

for(Configuration configuration : project.buildscript.configurations.findAll())
{
    copyJars(configuration)
    copyPoms(configuration)
}

---

Edit, December 2019: Three years later after a good bunch of upvotes over the whole time and now an explicit request, it appears this topic is still important not only to me.

To avoid link breakage, I will post the current version of my project skeleton setup right here.

The following assumes you are starting with a blank slate / empty folder that is supposed to be your future project directory. All paths I specify here are relative to the project root folder.

1. Get a gradle-wrapper.jar and place it in /gradle/wrapper. Place the accompanying gradlew and gradlew.bat run scripts in the project's root folder. Either grab those directly if you have them around somewhere, or install Gradle on your local system and run gradle wrapper in your project's root folder to generate these files. (You do not need to install Gradle locally if you can get the wrapper jar and scripts from somewhere else, and even if not, this is a one-time job. Once the wrapper is in place, there is no more need for a local Gradle installation.)
2. Grab a Gradle binary distribution package (file name like gradle-X.X-bin.zip) from somewhere and place it in /gradle/wrapper/.

3. Create (or modify an existing) /gradle/wrapper/gradle-wrapper.properties to contain the following:

   distributionBase=PROJECT
   distributionPath=gradle/wrapper/dists
   zipStoreBase=PROJECT
   zipStorePath=gradle/wrapper/dists
   distributionUrl=./gradle-X.X-bin.zip
   

   (The file name specified in distributionUrl obviously is just a placeholder and has to match the actual filename or the binary distribution package you used in step 2.) (You can alternatively set both distributionBase and zipStoreBase to GRADLE_USER_HOME to avoid having those temp files created in your project's folder; they will be created in your home folder instead.)

4. Make sure you exclude certain temporary files and directories from version control. I myself am using Git, so my example ignore-file is for Git. Adjust as needed if you are using a different version control system:

   .gradle/
   gradle/wrapper/dists/
   build/
   buildSrc/.gradle
   buildSrc/build/
   

5. Add the file /buildSrc/src/main/groovy/buildutils/OfflineMavenRepository.groovy with following contents (this is bmuschko's code linked above with all my proposed modifications applied to it):

   package buildutils
   
   import org.gradle.api.tasks.Input
   import org.gradle.api.tasks.Optional
   import org.gradle.api.tasks.OutputDirectory
   import org.gradle.api.tasks.TaskAction
   import org.gradle.api.DefaultTask
   import org.gradle.util.GFileUtils
   import org.gradle.api.artifacts.Configuration
   import org.gradle.api.artifacts.component.ModuleComponentIdentifier
   import org.gradle.maven.MavenModule
   import org.gradle.maven.MavenPomArtifact
   
   class OfflineMavenRepository extends DefaultTask {
       @OutputDirectory
       File repoDir = new File(project.projectDir, 'dependencies/maven')
   
       @TaskAction
       void build() {
           // Plugin/Buildscript dependencies
           for(Configuration configuration : project.buildscript.configurations.findAll())
           {
               copyJars(configuration)
               copyPoms(configuration)
           }
   
           // Normal dependencies
           for(Configuration configuration : project.configurations.findAll())
           {
               copyJars(configuration)
               copyPoms(configuration)
           }
       }
   
       private void copyJars(Configuration configuration) {
           configuration.resolvedConfiguration.resolvedArtifacts.each { artifact ->
               def moduleVersionId = artifact.moduleVersion.id
               File moduleDir = new File(repoDir, "${moduleVersionId.group.replace('.','/')}/${moduleVersionId.name}/${moduleVersionId.version}")
               GFileUtils.mkdirs(moduleDir)
               GFileUtils.copyFile(artifact.file, new File(moduleDir, artifact.file.name))
           }
       }
   
       private void copyPoms(Configuration configuration) {
           def componentIds = configuration.incoming.resolutionResult.allDependencies.collect { it.selected.id }
   
           def result = project.dependencies.createArtifactResolutionQuery()
               .forComponents(componentIds)
               .withArtifacts(MavenModule, MavenPomArtifact)
               .execute()
   
           for(component in result.resolvedComponents) {
               def componentId = component.id
   
               if(componentId instanceof ModuleComponentIdentifier) {
                   File moduleDir = new File(repoDir, "${componentId.group.replace('.','/')}/${componentId.module}/${componentId.version}")
                   GFileUtils.mkdirs(moduleDir)                
                   File pomFile = component.getArtifacts(MavenPomArtifact)[0].file
                   GFileUtils.copyFile(pomFile, new File(moduleDir, pomFile.name))
               }          
           }
       }
   }
   

6. Create your /build.gradle skeleton as follows:

   repositories {
       maven {
           url './dependencies/'
       }
   
       mavenLocal()
       mavenCentral()
       jcenter()
   }
   
   buildscript {
       repositories {
           maven {
               url './dependencies/'
           }
   
           mavenLocal()
           mavenCentral()
           jcenter()
       }
   }
   
   import buildutils.OfflineMavenRepository
   task offlineDependencies (type:OfflineMavenRepository) {
       repoDir = new File(project.projectDir, 'dependencies/')
   }
   

7. Add your additional online dependency repos, dependencies and other build logic to your build.gradle, then run ./gradlew offlineDependencies. This will create a folder structure within /dependencies/ and download and place all the dependency JARs inside. You only have to do this once to get the files there; once they are in place, you will only need to run this command again if your project's dependencies change (i.e. additional dependency or version change).

8. Check everything in version control and you should be good. Only requirement for building the project is a working Java JDK on your system. Gradle is contained within your project's directory (usable via the wrapper), so are all your Maven dependencies.