If Java is a general purpose language, and building a program is something that can be described using the Java language, why isn't this the best way to write build files and instead we use tools like Ant, Maven, and Gradle? Wouldn't that be more straightforward, and also remove the need to learn yet another programming language? (BTW - this question can also be applied to other languages, like C#)
-
7You might have a bit more interesting of a question as to "why aren't general purpose languages used for build languages" - I mean, C isn't a build language either. I'd also suggest looking at the ceremony around compiling a single .java file in Java– user40980Mar 25, 2014 at 20:42
-
8This could probably be generalized as "Why bother with DSLs?"– FrustratedWithFormsDesignerMar 25, 2014 at 20:48
-
2A better question might be; why are IDEs/compilers/tools so bad that build tools are needed in the first place.– BrendanMar 26, 2014 at 1:43
-
6@Brendan that's a non-question as build tools and IDEs serve different purposes.– jwentingMar 26, 2014 at 7:33
-
1Because the "general purpose" languages should never be used instead of the well-defined, simple domain-specific languages. And, if a need to learn "yet another programming language" concerns you, you should not really be programming, try some other trade.– SK-logicMar 26, 2014 at 12:38
6 Answers
Specific Tool for a Specific Purpose
Verbosity
General purpose languages are often too verbose. If I had to manage a build in Java, I'd be very depressed very quickly by the size of the beast. However, it could easily be manageable using a DSL written in Java. And to some extent that's how you can see Gradle (for Groovy) and Buildr (for Ruby).
Difficulty
General purpose languages are hard. Well I don't think programming is hard and can't be picked up by anyone, but the point is: your build engineer isn't necessarily your programmer!
Purpose
That's more or less at the intersection of difficulty and verbosity. A language is designed for a purpose, and here we're talking about something very specific. So why would you need or want to use a general purpose language? For builds, you need:
- branches and conditionals to handle separate configurations, environments, etc...
- a set of instructions to extract data from your environment,
- a set of instructions to produce deliverables.
You don't really need much more.
Sure it's annoying when your build system seems like it's not flexible enough for that one special use case you're after, but it's probably far better than the alternative for most people.
That's probably why there's a polarity between people who prefer declarative build systems over most programmable onces: I guess a developer might have a natural tendency to look for ways to break out of the box.
Wait, Do We Really Need a Tool?
Another related question would be: do we really need a build tool? Isn't the fact that they exist in all languages the sign that they fill a gap that shouldn't even be there in the first place?
Some languages don't necessarily require a build tool. For instance, most scripting languages don't need one and resolve at load time. Or take Go, whose compiler will handle everything for you, which is a nice counterpoint: what if a compiler like gcc suddenly didn't need a bunch of flags, a linker, and a makefile to die everything together? Or if javac didn't need a build.xml or pom.xml to tell him what to do? Shouldn't dependency management directly be part of the language's own tooling, as the dependencies are a part of the final program?
It surely seems like a much simpler approach for the user (the builder). Though one could argue they're just doing in under the hood and taking away your choice and opportunities to influence that build process (but then you'd hope such a tool allows compiler extensions and similar things). Plus we used to see the tools and the language as two separate things, so he might seem unpure to suddenly have them so tightly coupled.
I don't think the language you use to build your programs is sthe issue. It's the language you use to program and its core platform and tooling that should matter, and we're still making headway on that.
Personally, I've used make/gmake/autotools/pmk and been happy with them for C, and I started with Java when all we had was make, then ant, and now I'm generally preferring Maven over all these alternatives. Though I can see value in gradle, buildr and others, but I like the prevalence of maven so far, until a more significant shift occurs. Plus I like that it's rigid, but still leaves you the ability to work around that if necessary. That it's not easy is a good thing.
It's a build tool. Just learn to embrace it and don't fight it. It's a losing battle. Or at least a very very long one.
-
Really like your pragmatism. My question is out of curiosity. I use maven (having used make and ant in the past), and it does "black magic" which is sometimes good and sometimes bad. But it is still magic.– vainoloMar 26, 2014 at 16:51
-
1"Or if javac didn't need a build.xml or pom.xml to tell him what to do?" - heh. it doesn't and never did. Jul 2, 2015 at 7:53
-
@immibis: that's debatable. I wouldn't really enjoy building my projects at the office with only javac :) It certainly would require a fair bit of glue with either a Makefile or a shell script or at the very least shell aliases to put things together. Or a gigantic program with everything in one folder and all deps in the repo. Not really something I'd like! :) But that's another debate entirely, unrelated to the OP's question.– haylemJul 9, 2015 at 9:35
-
Integrating the build system with the language is also problematic for polyglot projects. If I use both language A and B and want to have a single build process for them, which language's build system do I use? May 19, 2017 at 11:02
-
@SebastianRedl So how does the problem get easier by introducing a third language? Just pick the language that works better for you. (And of course, if you need a third language, Java can also be it.) May 27, 2017 at 13:33
Java
is an imperative language, Ant
, Maven
, etc. are declarative languages:
We could define the difference as follows:
- Imperative programming: telling the "machine" how to do something, and as a result what you want to happen will happen.
- Declarative programming: telling the "machine" what you would like to happen, and let the computer figure out how to do it.1
Build languages tell the builder what should be done, from where it should be taken, etc. The engine which runs the build (which is written in an imperative language, like @ElliottFrisch has noted), reads these instructions, and fulfills them.
Declarative languages may seem more appropriate in build scenarios, since build tasks are generally the same all over, and it is considered more maintainable and readable in that form than in full-fledged code form.
-
3There's at least one build system that uses an imperative language though. Scons uses Python. Mar 25, 2014 at 21:50
-
Making it imperative is no harder than writing a class which can contain a list of dependencies, and a method to resolve those dependencies, though. I suspect there's another reason --- such as the JVM startup time, perhaps.– user51532Mar 26, 2014 at 0:14
-
2@Lee: Almost all of the build tools used in the Java world (Ant, Maven, Gradle, SBT, …) also typically run on the JVM (with the exceptions of Rake, Buildr or Scons, which can, but don't have to run on the JVM). Mar 26, 2014 at 2:50
-
6@vainolo "Most people don't really like Ant/Maven/Make" really? That's a bold statement! Mar 26, 2014 at 11:57
-
1@BenThurley I people liked make, why was ant created? and if ant was liked, why was maven? And now why are people using Gradle? But I agree that it is a bold statement, and it is only backed by "anecdotal" evidence of some tens of java developers– vainoloMar 26, 2014 at 16:43
If you look at the features of a typical build system you find:
- Lots of data: names, switches, settings, configuration items, strings, etc
- Lots of interaction with the environment: commands, environment variables
- A relatively straightforward "build engine" handling dependencies, threading, logging etc.
If you set out to write a series of build files using some language (Java/C#/Python/etc), by about the third or fourth iteration you would settle on (a) keeping most of the data and external commands as data in something like XML (b) writing the "build engine" in your favorite language.
You would also find it useful to treat some of the data in your XML as an interpreted language, to trigger various features in the build engine. You might also interpret some macros or perform string substitutions, in the data.
In other words, you would finish up with Make, or Ant, or Rake, or MsBuild. An imperative language for the things it does well, and data structures to describe what you want to do, now usually in XML.
A number of factors count against using Java/C++/C# in these cases.
Firstly, you'd have to compile your build script before you could run it to build your app. How would you specify any packages, flags, compiler versions, tool paths needed to build your build script? Certainly, you could come up with a way around it, but its much more straightforward to have either have a language that doesn't need that build step (e.g. python) or a language that your build tool natively understands.
Secondly, build files are data heavy whereas Java/C++/C# are much more geared towards writing code and algorithms. Java and friends don't have a very succinct representation of all the data you'd want to store.
Thirdly, Java and friends need a lot of boilerplate to be valid. The build file would have to be inside a method inside a class with all of its imports. When using a scripting language or custom language you can avoid all that boilerplate and just have the build details themselves.
Indeed! Why not use a powerful and expressive language for a problem that is more complex than people often (initially) think? Especially when the people facing the problem are already competent with such a language. (Building is programmers' own problem and best solved by programmers.)
I also asked myself this question years ago and decided that Java is a good language for defining builds, especially for Java projects. And, as a result, I started doing something about it.
DISCLAIMER: In this answer I am promoting iwant, a build system I am developing. But since this is an opinionated discussion anyway, I'm sure it's ok.
I won't elaborate on the benefits of Java (power and expressiveness) or iwant specifically. If you are interested, you can read more on the iwant page.
Instead I'll consider why Java (and other GPLs) is so readily dismissed as unsuitable for building. Many answers and comments here are good examples of such thinking. Let's consider some of the typical arguments:
"Java is imperative, but builds are best defined in a declarative way", they might say.
True. But when using a language as a metalanguage for an internal DSL, what really counts is its syntax. Even an imperative language like Java can be tricked to be declarative. If it looks and feels declarative, it is (for practical purposes) declarative. For example:
JacocoTargetsOfJavaModules.with()
.jacocoWithDeps(jacoco(), modules.asmAll.mainArtifact())
.antJars(TestedIwantDependencies.antJar(),
TestedIwantDependencies.antLauncherJar())
.modules(interestingModules).end().jacocoReport(name)
This is a real example from the demo project of iwant.
In fact, compare this to some supposedly declarative build systems that expose their users to such imperative verbs as "test" or "compile". The above declaration contains only nouns, no verbs. Compiling and testing are tasks implicitly handled by iwant in order to grant the user the nouns he/she wants. It's not the language. It's how you use it.
"Java is verbose"
Yes, a lot of Java code out there is verbose. But again, it's not the language, it's how you use it. If an implementation is verbose, just encapsulate it behind a nice abstraction. Many GPLs provide adequate mechanisms for this.
Just imagine the above Java snippet written in XML. Replace parentheses with angle brackets and move them around. And then duplicate every keyword as a closing tag! Java as a syntax is not verbose.
(I know, comparing to XML is like taking candy from a baby, but so many builds just happen to be defined in XML.)
"You'd have to compile your build script"
This is a valid point. Nevertheless, it's just a minor technical problem to be solved. I could have solved it by using beanshell or some other interpreter. Instead, I solved it by treating it as just another build problem and bootstrapping iwant with a simple shell or ant script that compiles and runs a simple Java bootstrapper.
"Java has boilerplate"
True. You have to import classes, you have to mention "public", "class" and so on. And here simple external DSLs score an initial easy win.
And if your project is so trivial that this boilerplate is significant, congratulations. Your problem isn't a difficult one and it really doesn't matter how you solve it.
But many projects need a lot more than compilation, coverage report and packaging. If the boilerplate of Java is acceptable for customers' problems, why not build problems? Why make shoes only for others' children?
One thing I don't think the other answers have addressed is that the limitations and transparency of these build languages is a huge part of what makes them useful. Let's take Maven, for example. Yes, it executes builds but it also defines dependencies. This allows for a Maven build to pull down those dependencies and look at their dependencies and so on and so forth.
Consider if this was done with Java directly. The build tool would see that there is a dependency. It would then need to pull down so other Java application and execute it to determine what it's dependencies are. But for Maven, it simply looks at the dependency declarations. The maven build file is transparent. A Turing complete language is inherently non-transparent and therefore are inferior for some purposes such as this one.
-
How does Gradle fit in this? It defines dependencies in a declarative style, using a general purpose language (Groovy). In a Groovy, JavaScript or Lisp based tool it's natural to use the language's compiler or interpreter to parse declarations, whether you just want to read them or 'execute' them (apply some function). That duality of code and data is not part of the generally accepted Java idiom, though not impossible. Turing completeness does not prevent good data representation. It does open the possibility your data will do something you didn't expect it to do.– joshpMay 19, 2017 at 16:39
-
@joshp I'm not familiar with Gradle. It's described as a DSL and things look rather declarative. Being based on Groovy doesn't necessarily mean it's equivalent to Groovy in it's power. You'd probably be in a better position to say if Gradle is in fact a Turing-complete language. The dependency examples I looked at seem fairly simple. Can you use arbitrary Groovy expressions in things like dependency declarations? May 19, 2017 at 17:14
-
Personally I don't even like transitive dependencies. Often they just cause surprises in the classpath (multiple incompatible versions of libraries). That's why maven has the exclude element, but the hassle is not just worth it. Explicit dependencies make life simpler. Still, transitive dependencies can be implemented with Java. I have done something like that with iwant by reusing build definitions of other projects. May 24, 2017 at 11:42
-
@VilleOikarinen I happen to mostly agree with you. I think it also causes a ton of bloat since there's no perceived cost to pulling in more dependencies even if you don't use them. However, in the context of this answer, I'm not commenting on the value or wisdom of that feature but on how the use of a DSL is useful in achieving it. May 24, 2017 at 14:36
-
1@VilleOikarinen I think we've kind of reached the end of this but I just want to clarify that I am not talking about pom/maven. This is an example of a build DSL but I don't think much of it. I use it begrudgingly and I think it's clunky. But what has pom so a dominant is the dependency declarations. I used Buildr for a while and it reads pom for dependencies but it doesn't use them for the build specifications. There are a number of tools that are not Java based that understand this pom but AFAIK, all they care about is the dependencies. Jun 2, 2017 at 13:52