25

I am reading C# Design Pattern Essentials. I'm currently reading about the iterator pattern.

I fully understand how to implement, but I don't understand the importance or see a use case. In the book an example is given where someone needs to get a list of objects. They could have done this by exposing a public property, such as IList<T> or an Array.

The book writes

The problem with this is that the internal representation in both of these classes has been exposed to outside projects.

What is the internal representation? The fact it's an array or IList<T>? I really don't understand why this is a bad thing for the consumer (the programmer calling this) to know...

The book then says this pattern works by exposing its GetEnumerator function, so we can call GetEnumerator() and expose the 'list' this way.

I assume this patterns has a place (like all) in certain situations, but I fail to see where and when.

5
  • 1
    Further reading: en.m.wikipedia.org/wiki/Law_of_Demeter
    – Jules
    Oct 8, 2015 at 13:51
  • 4
    Because, the implementation might want to change from using an array, to using a linked list without requiring changes in the consumer class. This would be impossible if the consumer knows the exact class returned.
    – MTilsted
    Oct 8, 2015 at 13:52
  • 11
    It's not a bad thing for the consumer to know, it's a bad thing for the consumer to rely on. I don't like the term information-hiding because it leads you to believe that the information is "private" like your password instead of private like the inside of a phone. The inner components are hidden because they are irrelevant to the user, not because they are some kind of secret. All you need to know is dial here, talk here, listen here. Oct 8, 2015 at 20:04
  • Suppose that we have a nice "interface" F that gives me knowledge (methods) of a, b and c. All is well and good, there can be many things that are different but just Fs to me. Think of the method as "constraints" or clauses of a contract F commits classes to doing. Suppose that we add a d though, because we need it. This adds an additional constraint, each time we do this we impose more and more on the Fs. Eventually (worst case) only one thing can be an F so we may as well not have it. And F constrains so much there's only one way to do it.
    – Alec Teal
    Oct 9, 2015 at 8:59
  • @captainman, what a wonderful comment. Yes I see why to 'abstract' many things, but the twist about knowing and relying is... Frankly.... Brilliant. And an important distinction that I didn't consider until reading your post. Oct 9, 2015 at 19:27

6 Answers 6

58

Software is a game of promises and privileges. It is never a good idea to promise more than you can deliver, or more than your collaborator needs.

This applies particularly to types. The point of writing an iterable collection is that its user can iterate over it - no more, no less. Exposing the concrete type Array usually creates many additional promises, e.g. that you can sort the collection by a function of your own choosing, not to mention the fact that a normal Array will probably allow the collaborator to change the data that's stored inside it.

Even if you think this is a good thing ("If the renderer notices that the new export option is missing, it can just patch it right in! Neat!"), overall this decreases the coherence of the code base, making it harder to reason about - and making code easy to reason about is the foremost goal of software engineering.

Now, if your collaborator needs access to a number of thingies so that they are guaranteed not to miss any of them, you implement an Iterable interface and expose only those methods that this interface declares. That way, next year when a massively better and more efficient data structure appears in your standard library, you'll be able to switch out the underlying code and benefit from it without fixing your client code everywhere. There are other benefits to not promising more than is needed, but this one alone is so big that in practice, no others are needed.

1
  • 12
    Exposing the concrete type Array usually creates many additional promises, e.g. that you can sort the collection by a function of your own choosing... - Brilliant. I just didn't think of it. Yes, it brings back an 'iteration' and only, an iteration! Oct 8, 2015 at 11:28
20

Hiding the implementation is a core principle of OOP and a good idea in all paradigms, but it especially important for iterators(or whatever they're called in that specific language) in languages that support lazy iteration.

The problem with exposing the concrete type of iterables - or even interfaces like IList<T> - is not in the objects that expose them, but in the methods that use them. For example, let's say you have a function for printing a list of Foos:

void PrintFoos(IList<Foo> foos)
{
    foreach (foo in foos)
    {
        Console.WriteLine(foo);
    }
}

You can only use that function to print lists of foo - but only if they implement IList<Foo>

IList<Foo> foos = //.....
PrintFoos(foos);

But what if you want to print every even-indexed item of the list? You'll need to create a new list:

IList<Foo> everySecondFoo = new List<T>();
bool isIndexEven = true;
foreach (foo; foos)
{
    if (isIndexEven)
    {
        everySecondFoo.Add(foo);
    }
    isIndexEven = !isIndexEven;
}
PrintFoos(everySecondFoo);

This is quite long, but with LINQ we can do it to a one-liner, which is actually more readable:

PrintFoos(foos.Where((foo, i) => i % 2 == 0).ToList());

Now, did you notice the .ToList() in the end? This converts the lazy query into a list, so we can pass it to PrintFoos. This requires an allocation of a second list, and two passes on the items(one on the first list to create the second list, and another on the second list to print it). Also, what if we have this:

void Print6Foos(IList<Foo> foos)
{
    int counter = 0;
    foreach (foo in foos)
    {
        Console.WriteLine(foo);
        ++ counter;
        if (6 < counter)
        {
            return;
        }
    }
}

// ........

Print6Foos(foos.Where((foo, i) => i % 2 == 0).ToList());

What if foos has thousands of entries? We will have to go through all of them and to allocate a huge list just to print 6 of them!

Enter Enumerators - C#'s version of The Iterator Pattern. Instead of having our function accept a list, we make it accept Enumerable:

void Print6Foos(Enumerable<Foo> foos)
{
    // everything else stays the same
}

// ........

Print6Foos(foos.Where((foo, i) => i % 2 == 0));

Now Print6Foos can lazily iterate over the first 6 items of the list, and we don't need to touch the rest of it.

Not exposing the internal representation is the key here. When Print6Foos accepted a list, we had to give it a list - something that supports random access - and therefore we had to allocate a list, because the signature does not guarantee that it'll only iterate over it. By hiding the implementation, we can easily create a more efficient Enumerable object that supports only what the function actually needs.

0
7

Exposing the internal representation is virtually never a good idea. It does not only make the code harder to reason about, but also harder to maintain. Imagine you have chosen any internal represenation - lets say an IList<T> - and exposed this internal. Anybody using your code may access the List and code may rely on the internal representation being a List.

For whatever reason you are deciding to change the internal representation to IAmWhatever<T> at some later point in time. Instead on simply changing the internals of your class you will have to change every line of code and method relying on the internal representation being of type IList<T>. This is cumbersome and prone to errors at best, when you are the only one using the class, but it may break other peoples code using your class. If you just exposed a set of public methods without exposing any internals you could have changed the internal representation without any line of code outside your class taking notice, working as if nothing has changed.

This is why encapsulation is one of the most important aspects of any nontrivial software design.

6

The less you say, the less you have to keep saying.

The less you ask, the less you have to be told.

If your code only exposes IEnumerable<T>, which supports only GetEnumrator() it can be replaced by any other code that can support IEnumerable<T>. This adds flexibility.

If your code only uses IEnumerable<T> it can be supported by any code that implements IEnumerable<T>. Again, there is extra flexibility.

All of linq-to-objects for example, depends only on IEnumerable<T>. While it fast-paths some particular implementations of IEnumerable<T> it does this in a test-and-use way that can always fallback to just using GetEnumerator() and the IEnumerator<T> implementation that returns. This gives it much more flexibility than if it was built on top of arrays or lists.

1
  • Nice clear answer. Very apt in that you keep it short, and therefore follow your advice in the first sentence. Bravo! Oct 14, 2015 at 8:23
1

A wording I like to use mirrors @CaptainMan 's comment: "It is fine for a consumer to know internal details. It is bad for a customer to need to know internal details."

If a customer has to type array or IList<T> in their code, when those types were internal details, the consumer must get them right. If they are wrong, the consumer may not compile, or even get wrong results. This yields the same behaviors as the other answers of "always hide the implementation details because you shouldn't expose them," but starts to dig at the advantages of not exposing. If you never expose an implementation detail, your customer can never put themselves in a bind by using it!

I like to think of it in this light because it opens up the concept of hiding the implementation to shades of meaning, rather than a draconian "always hide your implementation details." There are plenty of situations where exposing implementation is totally valid. Consider the case of real time device drivers, where the ability to skip past layers of abstraction and poke bytes into the right places can be the difference between making timing or not. Or consider a development environment where you don't have time to make "good APIs," and need to use things immediately. Often exposing underlying APIs in such environments can be the difference between making a deadline or not.

However, as you leave those special cases behind and look at the more general cases, it starts to become more and more important to hide implementation. Exposing implementation details is like rope. Used properly, you can do all sorts of things with it, like scaling tall mountains. Used improperly, and it can tie you in a noose. The less you know about how your product will be used, the more careful you need to be.

The case study that I see time and time again is one where a developer makes an API that isn't very careful about hiding implementation details. A second developer, using that library, finds that the API is missing some key features they would like. Rather than writing a feature request (which could have a turnaround time of months), they decide instead to abuse their access to the hidden implementation details (which takes seconds). This isn't bad in and of itself, but often the API developer never planned on that being part of the API. Later, when they improve the API and change that underlying detail, they find they are chained to the old implementation, because someone used it as part of the API. The worst version of this is where someone used your API to provide a customer-centric feature, and now your company's paycheck is tied to this flawed API! Simply by exposing implementation details when you did not know enough about your customers proved to be enough rope to tie a noose around your API!

7
  • 1
    Re: "Or consider a development environment where you don't have time to make 'good APIs,' and need to use things immediately": If you find yourself in such an environment, and for some reason don't want to quit (or aren't sure if you do), I recommend the book Death March: The Complete Software Developer's Guide to Surviving 'Mission Impossible' Projects. It has excellent advice on the subject. (Also, I recommend changing your mind about not quitting.)
    – ruakh
    Oct 9, 2015 at 4:09
  • @ruakh I have found it's always important to understand how to work in an environment where you don't have time to make good APIs. Frankly, as much as we love the ivory tower approach, real code is what actually pays the bills. The sooner we can admit that, the sooner we can start approaching software as a balance, balancing API quality with productive code, to match our exact environment. I've seen too many young developers trapped in a mess of ideals, not realizing they need to flex them. (I've also been that young developer.. still recovering from 5 years of "good API" design)
    – Cort Ammon
    Oct 9, 2015 at 5:19
  • 1
    Real code pays the bills, yes. Good API design is how you ensure that you will continue to be able to generate real code. Crappy code stops paying for itself when a project becomes unmaintainable and it becomes impossible to fix bugs without creating new ones. (And anyway, this is a false dilemma. What good code requires is not time so much as backbone.)
    – ruakh
    Oct 9, 2015 at 5:45
  • 1
    @ruakh What I have found is that it is possible to make good code without "good APIs." If given the opportunity, good APIs are nice, but to treat them as a necessity causes more strife than it's worth. Consider: the API for human bodies is horrendous, but we still think they're pretty good little feats of engineering =)
    – Cort Ammon
    Oct 9, 2015 at 6:56
  • The other example I'd give is the one which was the inspiration for the argument about making timing. One of the groups I work with had some device driver stuff they needed to develop, with hard realtime requirements. They had 2 APIs to work with. One was good, one was less so. The good API couldn't make timing because it hid the implementation. The lesser API exposed the implementation, and in doing so let you use processor-specific techniques to create a working product. The good API was politely put on the shelf, and they continued to meet timing requirements.
    – Cort Ammon
    Oct 9, 2015 at 7:04
0

You don't necessarily know what the internal representation of your data is - or may become in the future.

Assume you have a datasource which you represent as an array, you might iterate over it with a regular for loop.

Now say you want to refactor. Your boss has asked that the datasource be a database object. Furthermore he would like to have the option to pull data from an API, or a hashmap, or a linked list, or a spinning magnetic drum, or a microphone, or a realtime count of yak shavings found in Outer Mongolia.

Well if you have only one for loop you can modify your code easily to access the data object in the way it expects to be accessed.

If you have 1000 classes trying to access the data object, well now you have a big refactoring problem.

An iterator is a standard way to access a list based datasource. It's a common interface. Using it will make your code datasource agnostic.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.