Should developers be expected to compile an internal library before the actual program?

Question

Recently a senior developer that I work with made a case for requiring that developers get the latest version and compile as part of their project a major internal library. This stands in contrast the the counter argument that project teams should be working off a stable version that they get from an internal Maven repository; to which the developer argued that having the source code available on the developer machines saves time, as they can read the library's source code to determine if required functionality is available.

Does the senior developer have a valid argument? Or is requiring developers to read the library's source code run counter to the basic philosophy of encapsulation, and even having the library in the first place?

Answer 1

This senior developer's argument makes no sense to me.

They want to add overhead of constantly retrieving & compiling an internal library just so devs can occasionally read the source code? That's going to waste a lot more time than having devs go look at the source code only when they need to check if a feature is available.

This is a particularly bad idea if the library and the clients are being developed by different development groups and the library is being actively developed. The client developers won't have any insulation from instability in the library.

I don't see why source code can't be made available to the developers without forcing them to have it be part of their normal build.

Answer 2

The suggestion is

We can save time by client-side developers reading the library's source code to determine if required functionality is available

You could make an alternative suggestion

We can save time by someone documenting the library to indicate what functionality is available.

Answer 3

I wouldn't buy the argument that having the ability to peruse the source locally is a benefit, but if the library is in active development (possibly to add support for your project's needs) then I don't think it's unreasonable to require developers to download updates frequently (possibly several times a day). I think it makes better business sense to make a compiled (and unit-tested) binary available instead of requiring developers to compile from source, though.

Do you have the ability within your project to set up some kind of shared repository where the latest compiled version would be available? Ideally you'd want a CI server that did the fetch-and-build on a schedule, but even if it's just a network resource that one of the team leads is responsible for updating periodically it might help. Of course, this should be on the library team's plate, but obviously they're not interested, so you'll have to pick up their slack.

Answer 4

I used to work for a large software company who was constantly "dogfooding" their own software with internal business systems.

They saw this as another level of testing.

That I agree with, for a company, is a good thing.

I think forcing you to download, and compile the latest version is a step too far, unless the compilation step is an important part of your companies offering of selling, or even outsourcing, a library.

Answer 5

While having the source code available can be a benefit, if you have a CI build agent monitoring the repository, it makes WAY more sense to have that agent compile this library and copy it into dependent projects as an external, than to require the developers to run two different compilation steps when building their copy.

I'm currently working on a project that isn't hooked up to a build agent, that requires building a sub-application before building the main application. It's a serious pain in my posterior; to make a change to the sub-app, I must first build the entire project, then go to a sub-build folder, get the compiled product out of that and copy it to a different sub-folder, before building the entire project AGAIN to make sure the latest version of the sub-app is included in the build of the main app. This is NOT how it should be done; at the very least there should be an MSBuild script that will automate this process, and I would prefer that the build agent update the externals whenever new code is committed to the trunk.

Answer 6

Since so many people answered that it makes no sense to have everyone build an internal library, I'll present the opposite viewpoint which could justify the reasons:

1. You use the library a lot and there's no documentation. So everyone should have the source for reference. If this is a library that gets used very frequently, having the reference handy could be useful

   • Sure, one would say that they should work on documentation and more than likely your senior developer is aware of this fact. But let's face reality, a lot of times development teams end up with a ton of undocumented code, so when you need to know how it works, you go to the source. You shouldn't have to do it, but in the short term, this is the best alternative.
   • Usually the best people to put documentation in place are those that know the library the best. Unfortunately, those are the same people who are typically the busiest, so it's often not that easy for them to drop everything and start documenting.

2. When people start writing their own code that depends on the library and something in the library doesn't work, instead of throwing their arms in the air, if the library is built locally, it becomes very easy to step right inside the library code

   • This can occur because many libraries are far from perfect and while we all want to work with a "stable" release, there can still be issues.
   • Maybe you are getting bad results because of misunderstanding how an API should be used. Even with documentation, people often make wrong assumptions
   • Your senior guy is probably tired of new people (and sometimes there's a lot more new people than those that know the project inside and out) throwing their arms in the air whenever a crash/some other error seems to come from a library code. So instead of having to come over to your machine and then to the guy sitting next to you, they want the option of responding with these questions: 1) where exactly is the crash? what module/file/line? 2) have you debugged it? what have YOU found?
   • Your senior guys worked with this code base (application and your major library) long enough and potentially he may have noticed that when the code is already on the machine and is ready to be debugged and stepped through, it makes his life easier and gets people to learn the code base faster. So for that reason he forces you to take the upfront cost of building the library on your machine.

I'm not saying his decision is justified, just pointing out that a) question presented one side of the story and b) there could be plausible motives.

Answer 7

This kind testing would better be done indeed. Thing is though, it should be done by testers, not by developers. In that sense, it's neither your nor library developer's job.

_{From what you describe it sounds like there are no testers in the project - if this is the case, that's a management problem, and quite a serious one.}

...saves time as they can read the libraries source code to determine if required functionality is available

Quite lame reasoning. When most recent version library fails to compile with most recent version project, there could be multiple various reasons for that - just drilling into lib source code could be waste of time.

• What if library is OK and build failure was caused by the bug in project code? Or, what if build failure was caused by temporary incompatible change that is supposed to be corrected a day or two later? What if a build failure indicates a complicated integration issue that will take a week or month to address? For an integration issue, would using a prior version library make a workaround or not?
   
  Whatever the reason could be, doing preliminary analysis of the failure would mean wasting developer's time on a work that is supposed to be done by testers.

Another thing above reasoning misses is inevitable (and quite painful in my experience) productivity losses that follow when one has to break the flow by switching between development and QA activities.

---

When there are testers in the team, such things are really simple and can be handled much easier. What your "senior" developer cast at you is basically a draft testing requirement.

Upon every change made to the project or library, make sure that build is successful.

Steps to proceed from there are typical QA activities: clarify requirement details, design a formalized test scenario, negotiate on how to handle test failures.

• From SQA perspective, this is quite a routine task of designing, setting up and maintaining a pretty simple regression testing procedure that could be highly automated - probably up to the point that only manual activity would be creating and maintaining tickets in issue tracker and verification of fixes.

Answer 8

What the Sr Dev is suggesting makes little sense to me at best. It is nice to be able to browse sources, but there are better ways to do so.

What Artifact Repository are you using? You should be able to deploy a source jar for each version to live next to the compiled library. Most IDEs will then allow you to attach this to the compiled library for source browsing. Eclipse with the Maven Plugin will do this automatically.

If you need the very latest code, you can just deploy version snapshots.

Answer 9

This should simply happen in your build script:

1. check whether a new version is available. skip 2 otherwise.
2. download and compile it and run whatever tools you have to generated an API reference from the source code locally. show a change log.
3. build your app

I don't see why or how this is a problem. Also when something is missing in the reference, you can add it to the sources and push the changes. Of course it might seem a little scary that the library can change right under your feet, but if the library maintainers do their job properly, this is a good thing.