20

I used to be a Java developer for a long time, but recently, I joined a Haskell team. In the java world, if you have a large project, with several teams working on it, a common approach is to use an artifact server such as Maven to ease and speed-up the development. Numerous build tools, such as Ant, Maven, Gradle, can build the project and upload a jar file to the artifact server that can be used by the rest of the team without pain. Therefore, by splitting the project into smaller sub-projects, the build time is also drastically reduced.

On the Haskell side, we are using cabal to build the project. Our project takes about 10-15 minutes to build without optimization on. It takes a few hours if compiler optimization is turned on, which is painful.

I wonder, how we can do the same thing as we do in Java here. Is there an easy way to compile and upload the binary of the packages (libraries) to an artifact server and use the prebuilt binaries at the build time? I know that since Haskell generates machine code (rather than byte code in Java) there might be compatibility issues, but we can probably have different binaries for different architectures/OSs stored on the artifact server.

  • Not an answer to your question but, do you invoke "cabal build/install" with the -j option? – Daniel Díaz Carrete Mar 5 '15 at 21:30
  • Yes, but I don't see much speedup. CPU usage is around 15%-20% when building. I am not sure where the problem is: cabal, GHC, Test.Framework or the linker. – Oxy Mar 5 '15 at 22:13
  • According to this SO answer stackoverflow.com/questions/27173910/… the main reason is the native nature of Haskell executables. – Daniel Díaz Carrete Mar 6 '15 at 7:20
  • I am mostly talking about development. It is true that binary may cause incompatibility because of different OSs, CPU architectures and even compiler versions. But, why should I waste so much time when I am developing on the same machine, the same OS and the same compiler every day? – Oxy Mar 6 '15 at 18:46
  • It looks like Halcyon with a cache might be useful: halcyon.sh/reference/#private-storage-options – Lubomír Sedlář Mar 6 '15 at 19:09
2

You might consider using Nix, which is a general-purpose, cross-platform package manager with decent support for Haskell.

Nix has a custom programming language for defining packages (which just so happens to be pure, functional and lazy). Defining new packages and extending existing ones is pretty easy (eg. to alter dependencies, get the source from a different git repo, etc.).

Packages are identified by hashes, which include dependencies. Hence multiple versions, or the same version with different dependencies, can live side by side without conflict. Nix can look up the desired hash on a "binary cache" server, to see if that particular package with those particular dependencies has already been built; if so, it'll download the build product rather than compiling.

Currently, the nixpkgs repository includes most/all of Hackage, several GHC versions (7.10.1, 7.8.4 and some JS backends) and a cabal2nix utility which does a pretty good job of generating Nix packages from .cabal files. There's also the Nix-based "hydra" CI server, which you could use to trigger builds based on SCM commits.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.