3

I have a large codebase of Java/JSP files that needs to be compiled and checked for errors. The compilation time is long enough that we only run it once a day. However, I'd like it to be more frequent than that, say every 15 minute interval.

But I'm not able to figure out how to minimize the build time to allow this. Running the ANT script, downloading all files off Version Control, compiling JSP's into JAVAs and then compiling those JAVAs into Class files takes an hour and a half.

My first solution was to download a copy of the codebase and sync changes from the server (svn update) every 15 minutes, in order to reduce the downloading time. However, I have not been able to minimize the compilation time. I'm using Jasper to compile the JSPs and building all the files (into JAVA files) takes several minutes. Compiling the produced Java files takes another 15 minutes. To tackle this, I though about individually compiling the "updated" JSP files and copying them back to the JAVA source. This would take away the JSP->JAVA stage. However, ANT and Jasper haven't made it easy to compile individual files and also ensure that no new errors show up.

Should I settle for once a day build? Or is there a better solution that can be applied here?

Improve this question
14
  • 7
    Stand up a CI server.
    – Robert Harvey
    Mar 21, 2017 at 16:18
  • 2
    On another note, you probably don't need a complete compile every 15 minutes. You can almost certainly make do with partial compiles that take less time, or longer intervals.
    – Robert Harvey
    Mar 21, 2017 at 16:20
  • 1
    I would say that you will needsome huge refactoring work. Split the project into subprojects, create (stable) APIs for communication between differente modules, etc.
    – SJuan76
    Mar 21, 2017 at 16:26
  • 1
    @RobertHarvey That's what I thought going in, but I found that Ant and Jasper are not really friendly when it comes to "partial" build. I can supply it the root address of the project directory and everything's fine. The minute I do something else, it starts throwing errors and warnings everywhere.
    – DemCodeLines
    Mar 21, 2017 at 16:26
  • 1
    Why does it take so long to perform each of the steps? How big is your codebase? Does that include tests?
    – mmathis
    Mar 21, 2017 at 16:47

2 Answers 2

Reset to default
2

I did this type of thing at a previous company. We had a very large C application that took ages to do a complete build. 45 minute compile time, but we managed to get it down to less than 6.

Measure:

Firstly as you are doing I would measure each part of the process:

  1. Check out 15 minutes.
  2. Compiling Java branch takes 20
  3. Compiling JSP branch takes 40

Time: 75 minutes

However you state in the question that is takes an hour and a half so we have a missing 15 minutes, where is this is the build process ?

Running the ANT script, downloading all files off Version Control, compiling JSP's into JAVAs and then compiling those JAVAs into Class files takes an hour and a half.

So the first thing is to do an update of SVN rather than a checkout as in comments you suggest, this reduces the build time to 62 minutes. (Average 6 builds a day)

Next I would think look at the build scripts, is it possible to build your JAVA code or JSP in parallel. As it's legacy code it might not have been told to build over multiple CPU's. Our project was still sitting on a single CPU. You mention ANT it has the ability to build in parallel. Parallel Tasks

If it already doing this does writing them to disk pose an issue ? I ask this because if you are creating a lot of objects, the disk could be hitting it's bandwidth. Try an SSD drive for building. I mention this because it's a low cost solution to implement.

Other things to consider: You want fast feedback on a change, does it need to be full feedback ?

  1. Linting the code for faster feedback for some errors.
  2. Unit tests.
  3. CI server to distribute the load over several machines.

Long Term:

  1. Split the code out into multiple parts, do you have utilities that could be compiled independently, then included later ?
  2. PMD can show you where duplicate code is being used. Legacy projects normally have this issue where the same code is included in many places.
Improve this answer
1
  • 3
    Additional thing to check: do you have a running virus scanner that monitors every disk write operation and slows down build time?
    – k3b
    Mar 22, 2017 at 10:39
1

You consider modularising the code into smaller (logical) pieces. i.e. You might have some common modules like domain objects and DAOs, then another with services and business logic, then several with different views. Each module will have a fraction of the compile time of the original project.

Is it really JUST compilation that takes all the time or are you running tests also? If you're running tests also, make sure your unit tests are actually unit tests (no database/external connectivity) and consider moving integration/functional tests to another build process to be run less often/as needed.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.