12

This question about when to use private and when to use protected in classes got me to think. (I'll extend this question also to final classes and methods, since it is related. I'm programming in Java, but I think this is relevant to every OOP language)

The accepted answer sais:

A good rule of thumb is: make everything as private as possible.

And another one:

  1. Make all classes final unless you need to subclass them right away.
  2. Make all methods final unless you need to subclass and override them right away.
  3. Make all method parameters final unless you need to change them within the body of the method, which is kinda awkward most of the times anyways.

This is pretty straightforward and clear, but what if I'm mostly writing libraries (Open Source on GitHub) instead of applications?

I could name a lot of libraries and situations, where

  • A library got extended in a way the developers would never have thought of
  • This had to be done with "class loader magic" and other hacks because of visibility constraints
  • Libraries got used in a way they were not built for and the needed functionality way "hacked" in
  • Libraries couldn't be used because of a small issue (bug, missing functionality, "wrong" behavior) that could not be changed due to reduced visibility
  • An issue that could not be fixed led to huge, ugly and buggy workarounds where overriding a simple function (that was private or final) could have helped

And I actually started naming these until the question got too long and I decided to remove them.

I like the idea of not having more code than needed, more visibility than needed, more abstraction than needed. And this might work when writing an application for the end user, where the code is only used by those who write it. But how does this hold up if the code is meant to be used by other developers, where it is improbable that the original developer thought of every possible use case in advance and changes/refactors are difficult/impossible to make?

Since big open source libraries are not a new thing, what is the most common way of handling visibility in such projects with object-oriented languages?

migrated from stackoverflow.com Aug 23 '17 at 12:09

This question came from our site for professional and enthusiast programmers.

  • Possible duplicate of Why is Encapsulation considered a primary principle in OOP? – gnat Aug 23 '17 at 12:45
  • given that you ask about open source, it makes even less sense to bend proper coding principles in order to address issues you listed than closed source, simply because one can contribute needed corrections right into the library code or fork it and make their own version with whatever corrections they want to – gnat Aug 23 '17 at 12:51
  • 2
    my point is not about this but about your reference to open source making no sense in this context. I can imagine how pragmatic needs may justify deviation from strict principles in some cases (also known as accruing technical-debt) but from this perspective it doesn't matter whether code closed or open source. Or more precisely it matters in opposite direction than one you imagined here because code being open source can can make these needs less pressing than closed because it offers additional options to address these – gnat Aug 23 '17 at 13:55
  • 1
    @piegames: I fully agree to gnat here, the problems you scetched are much more likely to occur in closed source libs - if it is an OS lib with a permissive license, if the maintainers ignores a change request, one can fork the lib and change the visibility by oneself, if necessary. – Doc Brown Aug 23 '17 at 21:50
  • 1
    @piegames: I don't understand your question. "Java" is a language, not a lib. And if "your little open source library" has too strict visibility, extending visibility afterwards does normally not break backwards compatibility. Only the other way round. – Doc Brown Aug 23 '17 at 22:03
15

The unfortunate truth is that many libraries get written, not designed. This is sad, because a bit of prior thought can prevent a lot of problems down the road.

If we set out to design a library, there will be some set of anticipated use cases. The library might not satisfy all use cases directly, but may serve as part of a solution. So the library needs to be flexible enough to adapt.

The constraint is that it's usually not a good idea to take the source code of the library and modify it to handle the new use case. For proprietary libraries the source may not be available, and for open source libraries it may be undesirable to maintain a forked version. It may not be feasible to merge highly specific adaptions into the upstream project.

This is where the open–closed principle comes in: the library should be open to extension without modifying the source code. That does not come naturally. This must be an intentional design goal. There is a wealth of techniques that can help here, the classic OOP design patterns are some of them. In general, we specify hooks where user code can safely plug in to the library and add functionality.

Just making every method public or allowing every class to be subclassed is not sufficient to achieve extensibility. First of all, it is really difficult to extend the library if it's not clear where user could can hook into the library. E.g. overriding most methods is not safe because the base class method was written with implicit assumptions. You really need to design for extensibility.

More importantly, once something is part of the public API you can't take it back. You can't refactor it without breaking downstream code. Premature openness limits the library to a suboptimal design. In contrast, making internal stuff private but adding hooks if later there is need for them is a safer approach. While that is a sane way to tackle the long-term evolution of a library, this is unsatisfactory for users who need to use the library right now.

So what happens instead? If there is significant pain with the current state of the library, the developers can take all the knowledge about actual use cases that accumulated over time, and write a Version 2 of the library. It will be great! It will fix all those by-design bugs! It will also take longer than expected, in many cases fizzling out. And if the new version is very dissimilar to the old version, it might be hard to encourage users to migrate. You're then left maintaining two incompatible versions.

  • So I need to add hooks for extension because just making it public/overridable is not enough. And I also need to think about when to release changes/new API because of backwards compatibility. But what about the visibility of methods in special? – piegames Aug 24 '17 at 14:54
  • @piegames With visibility you decide which parts are public (part of your stable API) and which parts are private (subject to change). If someone circumvents that with reflection, it's their problem when that feature breaks in the future. By the way, extension points are often in a form of a method that can be overridden. But there's a difference between a method that can be overridden, and a method that is intended to be overridden (see also the Template Method Pattern). – amon Aug 25 '17 at 10:42
8

Every public and extensible class/method is a part of your API that must be supported. Limiting that set to a reasonable subset of the library allows the most stability and limits the number of things that can go wrong. It's a management decision (and even OSS projects are managed to a degree) based on what you can reasonably support.

The difference between OSS and closed source is that most people are trying to create and grow a community around the code so that it's more than one person maintaining the library. That said, there are a number of management tools available:

  • Mailing lists discuss user needs and how to implement things
  • Issue tracking systems (JIRA or Git issues, etc.) track bugs and feature requests
  • Version control manages the source code.

In mature projects, what you'll see is something along these lines:

  1. Someone wants to do something with the library it wasn't originally designed to do
  2. They add a ticket to the issue tracking
  3. The team may discuss the issue in the mailing list or in the comments, and the requester is always invited to join the discussion
  4. The API change is accepted and prioritized or rejected for some reason

At that point, if the change was accepted but the user wants to accelerate it getting fixed, they can do the work and submit either a pull request or a patch (depending on the version control tool).

No API is static. However it's growth has to be shaped in some way. By keeping everything closed down until there is a demonstrated need to open things up, you avoid getting the reputation of a buggy or unstable library.

  • 1
    Fully agree, and I have sucessfully practiced the change request process you scetched for 3rd party closed-source libs as well, as well as for open-source libs. – Doc Brown Aug 23 '17 at 21:45
  • In my experience, even small changes in small libraries are a lot of work (even if it's just to convince the others) and might take some time (to wait for the next release if you can't afford using a snapshot until then). So this clearly isn't an option for me. I'd be interested though: are there any bigger libraries (in GitHub) that really use that concept? – piegames Aug 24 '17 at 14:40
  • It is always a lot of work. Almost every project I've contributed to has a process similar to that. Back in my Apache days we might discuss something for days because we were passionate about what we created. We knew a lot of people were going to use the libraries, so we had to discuss things like whether the proposed change was going to break the API, is the proposed feature worth it, when should we do it? etc. – Berin Loritsch Aug 24 '17 at 14:47
0

I'll reword my response since it seems it struck a nerve with a few people.

class property/method visibility has nothing to do with security nor openness of source.

The reason why visibility exists, is because objects are fragile to 4 specific problems:

  1. concurrency

If you build your module unencapsulated, then your users will get used to altering module state directly. This works fine in a single threaded environment, but once you even think about adding threads; you will be forced to make the state private and use locks/monitors along with getters and setters that make other threads wait for the resources, rather than racing on them. This means your users programs won't work anymore because private variables cannot be accessed in a conventional way. This can mean you need a lot of rewrites.

The truth is that it's much easier to code with a single threaded runtime in mind, and private keyword allows you to simply add the keyword synchronized, or a few locks, and your users' code won't break if you encapsulated it from the beginning.

  1. Help prevent users shooting themselves in the foot/streamline use of the interface. In essence, it helps you control the invariants of the object.

Every object has a bunch of things it requires to be true in order to be in consistent state. Unfortunately, these things live in client visible space because it's expensive to move each object into its own process and talk to it through messages. This means that it is very easy for an object to crash the whole program if the user has full visbility.

This is unavoidable, but you can prevent accidentally putting an object into inconsistent state by making an interface closure over its services that prevent accidental crashes by only allowing the user to interact with the object's state through a carefully crafted interface that makes the program much more robust. This doesn't mean the user can't intentionally corrupt the invariants, but if they do, it's their client that crashes, all they have to do is restart the program(the data you want to protect shouldn't be stored on client side).

Another nice example where you can improve usability of your modules is to make the constructor private; because if the constructor throws an exception, it will kill the program. One lazy approach to solving this is to make the constructor throw a compile time error you that you can't construct it unless it's in a try/catch block. By making the constructor private, and adding a public static create method, you can have the create method return null if it fails to construct it, or take a callback function to handle the error with, making the program more user friendly.

  1. Scope pollution

Many classes have a lot of state and methods and it is easy to get overwhelmed trying to scroll through them; Many of these methods are just visual noise such as helper functions, state. making variables and methods private helps reduce scope pollution and make it easier for the user to find the services they are looking for.

In essence, it lets you get away with having helper functions inside the class rather than outside the class; without visibility control without distracting the user with a bunch of services that the user should never use, so you can get away with breaking down methods into a bunch of helper methods(although it will still pollute your scope, but not the user's).

  1. being tied to dependencies

A well crafted interface can hide it's internal databases/windows/imaging that it depends on to do its work, and if you want to change to another database/another windowing system/another imaging library, you can keep the interface the same and the users won't notice.

On the other hand, if you don't do this, you can easily fall into making it impossible to change your dependencies, because they are exposed, and code relies on it. With a system big enough, the cost of migrating can become unaffordable, whereas a encapsulating it can protect well behaving client users from future decisions to swap out dependencies.

  • 1
    "No point in hiding anything" -- then why even think about encapsulation? In many contexts reflection does require special privileges. – Frank Hileman Aug 23 '17 at 18:24
  • You think about encapsulation because it gives you breathing space while developing your module, and reduce the likelihood of misuse. For example, if you have 4 threads modifying a class's internal state directly, it will easily cause problems, whereas making the variable private, encourages the user to use the public methods to manipulate the world state, which can use monitors/locks to prevent issues. This is the only real benefit of encapsulation. – Dmitry Aug 23 '17 at 18:42
  • Hiding things for the sake of security is an easy way to make a design where you end up with having to have holes in your api. A good example of this is multi-document applications, where you have many toolboxes, and many windows with subwindows. If you go nuts on encapsulation, you will end up having a situation where to draw something on one document, you have to ask the window to ask the inner document to ask the inner document to ask the inner document to put a request to draw something and invalidate its context. If the client side wants to play with client side, you can't prevent them. – Dmitry Aug 23 '17 at 19:38
  • OK that makes more sense, though security can be achieved via access control if the environment supports that, and it was one of the original goals in OO language design. Also you are promoting encapsulation and saying don't use it at the same time; a bit confusing. – Frank Hileman Aug 24 '17 at 19:49
  • I never meant not to use it; I meant don't use it for the sake of security; use it strategically to improve the experience of your users and to give yourself a smoother development environment. my point is that it has nothing to do with security nor openness of source. Client side objects are by definition vulnerable to introspection, and moving them out of user process space makes unencapsulated things equally unaccessible as encapsulated. – Dmitry Aug 24 '17 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.