20

In Java, C# and many other strongly-typed, statically checked languages, we are used to write code like this:

public void m1() { ... }
protected void m2() { ... }
private void m2() { ... }
void m2() { ... }

Some dynamically checked languages don't provide keywords to express the level of "privateness" of a given class member and rely on coding conventions instead. Python for example prefixes private members with an underscore:

_m(self): pass

It can be argued that providing such keywords in dynamically checked languages would add little use since it is only checked at runtime.

However, I can't find a good reason to provide these keywords in statically checked languages, either. I find the requirement to fill my code with rather verbose keywords like protected both annoying and distracting. So far, I have not been in a situation where a compiler error caused by these keywords would have saved me from a bug. Quite in contrary, I have been in situations where a mistakenly placed protected prevented me from using a library.

With this in mind, my question is:

Is information hiding more than a convention between programmers used to define what is part of the official interface of a class?

Can it be used to secure a class' secret state from being attacked? Can reflection override this mechanism? What would make it worthwhile for the compiler to enforce information hiding?

11
  • 10
    Sorry, Eric Lippert is on holiday :)
    – Benjol
    Aug 5, 2011 at 13:37
  • Your question is a bit difficult to understand. Are you basically just asking if someone can access a private member from outside? Aug 5, 2011 at 13:47
  • 1
    Related question: programmers.stackexchange.com/questions/92380/…
    – user281377
    Aug 5, 2011 at 13:49
  • @ironcode: I can tell that I somehow don't express myself clear enough from some of the answers. Can you tell what specifically makes it hard to understand? I'll try to clarify.
    – blubb
    Aug 5, 2011 at 13:57
  • 2
    "It is nonsense to provide information at compile time and then only check it at runtime" -- really? Better tell that to Microsoft because that's what a significant chunk of the current .NET framework is based on: selectively opting-in or opting-out of static typing. I don't think this is really what you mean though; the question repeatedly talks about "static" vs. "dynamic" checking, but those are characteristics of the type system; what's really at issue here is compile-time vs. run-time binding.
    – Aaronaught
    Aug 5, 2011 at 14:12

11 Answers 11

4

I'm studying for the Java certification and a whole bunch of it concerns that, how to manage access modifiers. And they make sense and they should be used properly.

I've worked with python and, during my learning journey, I've heard that in python, the convention is there because people, who work with it, should know what it means and how to apply it. That said, in _m(self): pass the underscore would alert me to not mess around with that field. But will everybody follow that convention? I work with javascript and I must say, they don't. Sometimes I need to check a problem and the reason was that the person was doing something that he wasn't supposed to do...

read this discussion regarding the python leading underscore

Is information hiding more than a convention between programmers used to define what is part of the official interface of a class?

From what I said, Yes.

Can it be used to secure a class' secret state from being attacked? Can reflection override this mechanism? Are there any other advantages provided by a more formal mechanism enforced by the compiler?

Yes it can be used to secure a class' secret state and it should be not only used for that but to prevent users from messing with the objects state in order to change their behaviors to something that they think should be the way the object should have been working. You, as a developer, should plan and think about it and design your class in a way that makes sense, in a way that its behavior won't be tampered. And the compiler, as a good friend, will help you keeping the code the way you designed it enforcing the access policies.

EDITED Yes, reflection can, check the comments

The last question is an interesting one and I'm willing to read the answers concerning it.

5
  • 3
    Reflection usually can bypass the access protection. For Java and C#, they both can access private data.
    – Mark H
    Aug 5, 2011 at 14:02
  • Thanks! The answers here: stackoverflow.com/questions/1565734/… explain it!
    – wleao
    Aug 5, 2011 at 14:10
  • 1
    Javascript culture is very different from python culture. Python have pep-08 and almost all python developers follow these conventions. PEP-08 violations are often considered as a bug. So I think javascript/php/perl/ruby experience translates very little into python experience in this aspect: _private works very well in python. Aug 8, 2011 at 19:01
  • 1
    In some (but not all) programming languages, information hiding can be used to protect against hostile code. Information hiding cannot be used to protect against hostile users.
    – Brian
    May 14, 2012 at 17:09
  • 2
    @Mike if PEP-08 violations are so often "considered as a bug", and no Python dev would dream of not following the conventions, then you may as well have them enforced by the language! Why do menial work when the language can do it automatically?
    – Andres F.
    Aug 27, 2012 at 2:55
27

The "private" access specifier is not about the compiler error it generates the first time you see it. In reality it's about preventing you from accessing something that is still subject to change when the implementation of the class holding the private member changes.

In other words, not allowing you to use it when it's still working prevents you from accidentally still using it when it's no longer working.

As Delnan remarked below the prefix convention discourages accidental use of members that are subject to change as long as the convention is followed and understood correctly. For a malicious (or ignorant) user it does nothing to stop them from accessing that member with all the possible consequences. In languages with built-in support for access specifiers this does not happen in ignorance (compiler error), and stands out like a sore thumb when malicious (strange constructions to get to the private member).

The "protected" access specifier is a different story - don't think of this as simply "not quite public" or "a lot like private". "Protected" means that you will probably want to use that functionality when you derive from the class containing the protected member. The protected members are part of the "extension interface" that you will use to add functionality on top of existing classes without changing those existing classes themselves.

So, short recap:

  • public: Use safely on instances of the class, the purpose of the class, will not change.
  • protected: To be used when extending (deriving from) the class - may change if the implementation has to change drastically.
  • private: Do not touch! May change at will to provide a better implementation of the expected interfaces.
7
  • 3
    More importantly, any (easily recognizable, which is the case with e.g. leading underscores) not enforced convention for "implementation detail, subject to change" also avoids accidential use a private members. In Python, for instance, the only time a sane programmer writes code using a private member from the outside (and even then it's frowned up) is when it's nearly unavoidable, he knows exactly what he does, and accepts the potential breakage in future versions. That is, a situation in which e.g. Java programmers would use reflection.
    – user7043
    Aug 5, 2011 at 13:41
  • 3
    @Simon Stelling - no, I am saying that the original implementor of the class that made something private is telling you "don't use this, any behavior here may change in future". For example, a private member variable of a class that first contains a distance between two points may later contain the distance between two points squared for performance reasons. Anything that would have relied on the old behavior would break silently. Aug 5, 2011 at 13:44
  • 1
    @MadKeithV: Then how is a language-level keyword saying "don't touch because ..." different from a convention meaning the same thing? Only the compiler enforcing it, but then we're back to ye olde static vs. dynamic discussion.
    – user7043
    Aug 5, 2011 at 13:48
  • 7
    @Simon Stelling (and Delnan) - to me that's kind of like asking "how is an engine limiter different from a speed limit sign". Aug 5, 2011 at 13:51
  • 1
    I'm well aware of the difference and I never said it wasn't there. I just find that your answer and most of your follow-up comments merely describe the kinds of annotations to members and methods that can be made with conventiones or language-level keywords and leave the actual difference (presence of lanugage-level enforcement) to be inferred by the reader.
    – user7043
    Aug 5, 2011 at 14:59
11

If you are writing code that will be consumed by someone else, then information hiding can provide a much easier to undertand interface. "Someone else" could be another developer on your team, developers consuming an API you wrote commercially, or even your future self who "just can't remember how the dang thing works". It is way easier to work with a class that has only 4 methods available than one that has 40.

4
  • 2
    I totally agree, but this doesn't answer my question. Whether I use _member or private member in my class is negligible, as long as the other programmer understands that he should only look at public or non-prefixed members.
    – blubb
    Aug 5, 2011 at 13:39
  • 6
    Yes, but the public documentation of a class written in a dynamic language's doesn't throw those 36 private parts at you either (at least not by default).
    – user7043
    Aug 5, 2011 at 13:42
  • 3
    @Simon The statement that it is "just a convention" is misleading, since it has real benefits. "Just a convention" is something like curly braces on the next line. A convention with benefits is something like using meaningful names, which I would say is totally different. Does setting something to private stop someone from seeing you "secret methods"? No, not if they are determined. Aug 5, 2011 at 13:51
  • 1
    @Simon- also, only exposing methods that are safe to use increases the likely hood that the user will not break something when trying to use the class. There could easily methods that could crash a webserver if called a thousand times. You do not want consumers of you API to be able to that. Abstraction is more than just a convention. Aug 5, 2011 at 13:55
4

I would suggest that for background, you start by reading about class invariants.

An invariant is, to make a long story short, an assumption about the state of a class which is supposed to remain true throughout the lifetime of a class.

Let's use a really simple C# example:

public class EmailAlert
{
    private readonly List<string> addresses = new List<string>();

    public void AddRecipient(string address)
    {
        if (!string.IsNullOrEmpty(address))
            addresses.Add(address);
    }

    public void Send(string message)
    {
        foreach (string address in addresses)
            SendTo(address, message);
    }

    // Details of SendTo not shown
}

What's going on here?

  • The member addresses is initialized on class construction.
  • It's private, so nothing from the outside can touch it.
  • We've also made it readonly, so nothing from the inside can touch it after construction (this isn't always correct/necessary, but it's useful here).
  • Therefore, the Send method can make an assumption that addresses is never going to be null. It doesn't have to perform that check because there is no way that the value can be changed.

If other classes were allowed to write to the addresses field (i.e. if it were public), then this assumption would no longer be valid. Every single other method in the class that depends on that field would have to start doing explicit null checks, or risk crashing the program.

So yes, it's a lot more than a "convention"; all of the access modifiers on class members collectively form a set of assumptions about when and how that state can be changed. Those assumptions are subsequently incorporated into dependent and interdependent members and classes so that programmers don't have to reason about the entire state of the program at the same time. The ability to make assumptions is a critical element of managing complexity in software.

To these other questions:

Can it be used to secure a class' secret state from being attacked?

Yes and no. Security code, like most code, is going to rely on certain invariants. Access modifiers are certainly helpful as signposts to trusted callers that they aren't supposed to mess with it. Malicious code isn't going to care, but malicious code doesn't have to go through the compiler either.

Can reflection override this mechanism?

Of course it can. But reflection requires the calling code to have that privilege/trust level. If you're running malicious code with full trust and/or administrative privileges, then you've already lost that battle.

What would make it worthwhile for the compiler to enforce information hiding?

The compiler already does enforce it. So does the runtime in .NET, Java, and other such environments - the opcode used to call a method will not succeed if the method is private. The only ways around that restriction require trusted/elevated code, and elevated code could always just write directly to the program's memory. It's enforced as much as it can be enforced without requiring a custom operating system.

2
  • 1
    @Simon: We need to clarify "ignorance" here. Prefixing a method with _ states the contract, but doesn't enforce it. That may make it hard to be ignorant of the contract, but it does not make it hard to break the contract, especially not when compared with tedious and often restrictive methods like Reflection. A convention says, "you shouldn't break this". An access modifier says, "you can't break this". I am not trying to say that one approach is better than the other - they are both fine depending on your philosophy, but they are nevertheless very different approaches.
    – Aaronaught
    Aug 5, 2011 at 17:49
  • 2
    How about an analogy: A private/protected access modifier is like a condom. You don't have to use one, but if you're not going to, then you'd better be damn sure about your timing and your partner(s). It will not prevent a malicious act, and might not even work every time, but it will definitely lower the risks from carelessness, and do so far more effectively than saying "please be careful". Oh, and if you have one, but don't use it, then don't expect any sympathy from me.
    – Aaronaught
    Aug 5, 2011 at 18:00
3

Information hiding is much more than just a convention; trying to go around it can actually break the functionality of the class in many cases. For instance, it's pretty common practice to store a value in a private variable, expose it using a protected or public property of the same time, and in the getter, check for null and do any initialization that's necessary (i.e., lazy loading). Or store something in a private variable, expose it using a property, and in the setter, check to see if the value changed and fire PropertyChanging/PropertyChanged events. But without seeing the internal implementation, you'd never know all that's going on behind the scenes.

3

Information hiding evolved from a top-down design philosophy. Python has been called a bottom-up language.

Information hiding is enforced well at the class level in Java, C++ and C#, so it's not really a convention at this level. It's very easy to make a class a "black box", with public interfaces and hidden (private) details.

As you pointed out, in Python it's up to programmers to follow the convention of not using what is intended to be hidden, since everything is visible.

Even with Java, C++ or C#, at some point information hiding becomes a convention. There are no access controls at the highest levels of abstraction involved in more complex software architectures. For example, in Java you can find the use of ".internal." package names. This is purely a naming convention because it's not easy for this kind of information hiding to be enforced via package accessibility alone.

One language that strives to define access formally is Eiffel. This article points out some other information-hiding weaknesses of languages such as Java.

Background: Information hiding was proposed in 1971 by David Parnas. He points out in that article that use of information about other modules can "disastrously increase the connectivity of the system structure." According to this idea, lack of information hiding can lead to tightly coupled systems that are hard to maintain. He continues with:

We wish to have the structure of the system determined by the designers explicitly before programming begins, rather than inadvertently by a programmer's use of information.

1
  • 1
    +1 for the quote "programmer's use of information", that's it exactly (although you have an extra comma at the end).
    – jmoreno
    Aug 27, 2012 at 2:06
2

Ruby and PHP have it and enforce it at runtime.

The point of information hiding is actually information showing. By "hiding" internal details, the purpose becomes apparent from an outside perspective. There are languages, which embrace this. In Java, access defaults to package internal, in haXe to protected. You explicitly declare them public to expose them.

The whole point of this is to make your classes easy to use by exposing only a highly coherent interface. You want the rest to be protected, so that no clever guy comes and messes with your internal state to trick your class into doing what he wants.

Also, when access modifiers are enforced at runtime, you can use them to enforce a certain level of security, but I don't think this is a particularly good solution.

4
  • 1
    If your library is being used in the same company in which you're working you will care... If he crashes his app you will have a hard time fixing it.
    – wleao
    Aug 5, 2011 at 14:13
  • 1
    Why pretend to wear a safety belt when you could wear one? I'd rephrase the question and ask, is there any reason for a compiler to not enforce it, if all information is available to it. You don't wanna wait until you have a crash to find out whether that safety belt was worth wearing.
    – Mark H
    Aug 5, 2011 at 14:25
  • 1
    @Simon: why clutter an interface with stuff you don't want people to use. I hate that C++ requires every member to be present in the class definition in a header file (I understand why). Interfaces are for other people and should not be littered with things they can't use.
    – phkahler
    Aug 5, 2011 at 14:26
  • 1
    @phkahler: pydoc removes _prefixedMembers from the interface. Cluttered interfaces have nothing to do with the choice of a keyword checked by the compiler.
    – blubb
    Aug 5, 2011 at 14:30
2

Acces modifiers can definitely do something that convention can't.

For example, in Java you cannot access private member/field unless you use reflection.

Therefore if I write the interface for a plugin and correctly deny rights to modify private fields through reflection (and to set security manager :) ), I can send some object to functions implemented by anyone and know that he cannot access its private fields.

Of course there can be some security bugs that would allow him to overcome this, but this is not philosophically important (but in practice it definitely is).

If the user runs my interface in his environment, he has control and thus can circumvent access modifiers.

2

I have not been in a situation where a compiler error caused by these keywords would have saved me from a bug.

It's not so much to save application authors from bugs as to let library authors decide which parts of their implementation they're committing to maintaining.

If I have a library

class C {
  public void foo() { ... }

  private void fooHelper() { /* lots of complex code */ }
}

I may want to be able to replace foo's implementation and possibly change fooHelper in radical ways. If a bunch of people have decided to use fooHelper despite all the warnings in my documentation then I may not be able to do that.

private lets library authors break libraries into manageable sized methods (and private helper classes) without the fear that they'll be forced to maintain those internal details for years.


What would make it worthwhile for the compiler to enforce information hiding?

On a side note, in Java private is not enforced by the compiler, but by the Java bytecode verifier.


Can reflection override this mechanism? What would make it worthwhile for the compiler to enforce information hiding?

In Java, not only reflection can override this mechanism. There are two kinds of private in Java. The kind of private that prevents one outer class from accessing another outer class's private members which is checked by the bytecode verifier, but also privates that are used by an inner class via a package-private synthetic accessor method as in

 public class C {
   private int i = 42;

   public class B {
     public void incr() { ++i; }
   }
 }

Since class B (really named C$B) uses i, the compiler creates a synthetic accessor method that allows B to access C.i in a way that gets past the bytecode verifier. Unfortunately, since ClassLoader allows you to create a class from a byte[], it's fairly simple to get at the privates that C has exposed to inner classes by creating a new class in C's package which is possible if C's jar has not been sealed.

Proper private enforcement requires coordination between the classloaders, bytecode verifier, and the security policy which can prevent reflective access to privates.


Can it be used to secure a class' secret state from being attacked?

Yes. "Secure decomposition" is possible when programmers can collaborate while each preserving the security properties of their modules -- I don't have to trust the author of another code module not to violate the security properties of my module.

Object Capabilities languages like Joe-E use information hiding and other means to make secure decomposition possible:

Joe-E is a subset of the Java programming language designed to support secure programming according to object-capability discipline. Joe-E is intended to facilitate construction of secure systems, as well as to facilitate security reviews of systems built in Joe-E.

The paper linked from that page gives an example of how private enforcement makes secure decomposition possible.

Providing secure encapsulation.

Figure 1. An append-only logging facility.

public final class Log {
  private final StringBuilder content;
  public Log() {
    content = new StringBuilder();
  }
  public void write(String s) {
    content.append(s);
  }
}

Consider Fig. 1, which illustrates how one might build an append-only log facility. Provided that the rest of the program is written in Joe-E, a code reviewer can be confident that log entries can only be added, and cannot be modified or removed. This review is practical because it requires only inspection of the Log class, and does not require review of any other code. Consequently, verifying this property requires only local reasoning about the logging code.

1

Information hiding is one of the primary concerns of good software design. Check out any of Dave Parnas' papers from the late '70s. Basically, if you can't guarantee that your module's internal state is consistent, you can't guarantee anything about its behavior. And the only way you can guarantee its internal state is to keep it private and only allow it to be changed by means you provide.

1

By making protection a part of the language, you gain something: reasonable assurance.

If I make a variable private, I have reasonable assurance that it will be touched only by code within that class or explicitly declared friends of that class. The scope of code that could reasonably touch that value is limited and explicitly defined.

Now are there ways to get around syntactic protection? Absolutely; most languages have them. In C++, you can always cast the class to some other type and poke at its bits. In Java and C#, you can reflect yourself into it. And so forth.

However, doing this is hard. It's obvious that you're doing something you ought not be doing. You cannot do it by accident (outside of wild writes in C++). You must willingly think, "I'm going to touch something that I was told not to by my compiler." You must willingly do something unreasonable.

Without syntactic protection, a programmer can just accidentally screw things up. You have to teach the user a convention, and they must follow that convention every time. If they don't, the world becomes very unsafe.

Without syntactic protection, the onus is on the wrong people: the many people using the class. They must follow the convention or unspecified badness will occur. Scratch that: unspecified badness may occur.

There is nothing worse than an API where, if you do the wrong thing, everything might work anyway. That provides false reassurance to the user that they have done the right thing, that everything is fine, etc.

And what of new users to that language? Not only do they have to learn and follow actual syntax (enforced by the compiler), they now must follow this convention (enforced by their peers at best). And if they don't, then nothing bad may happen. What happens if a programmer doesn't understand why the convention exists? What happens when he says "screw it" and just pokes at your privates? And what does he think if everything keeps working?

He thinks that the convention is stupid. And he won't follow it ever again. And he will tell all his friends not to bother too.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.