I was reading up on design patterns, and I read that the prototype design pattern does away with excessive subclassing.
Why is subclassing bad? What advantage would using a prototype bring about over subclassing?
Why is subclassing too much bad
"Too much" is a judgement call; but take it from me it is bad. The code I work with has DEEP inheritance and the code is just damn hard to read, understand, follow, trace, debug, etc. etc. It is effectively impossible to write test code for this stuff.
Prototype pattern does away with subclassing
Or does the question mean "does away with too much subclassing?". This pattern calls for cloning a "template" object to avoid subclassing, at least at that point. There's no rule that says the "template" can't be a subclass.
Favor composition over inheritance
This idea here also includes delegation and aggregation. Following this heuristic means your software tends to be more flexible, easier to maintain, extend, and reuse.
When a class is composed of parts you can substitute those parts at runtime. This has a profound effect on testability.
Testing is easier. You can use fake parts (i.e. "mocks", "doubles" and other testing-talk). Our code's deep inheritance means we must instantiate the entire hierarchy to test any bit of it. In our case that is not possible without running the code in it's real environment. For example we need a database in order to instantiate business objects.
Changes come with side effects and uncertainty - The more "base" the class the more widespread the effects, for good or bad. There may be desired changes you dare not make due to side effects uncertainty. Or a change that is good for some place in our inheritance chain is bad for another. This is certainly my experience.
I think one of the reasons this has been viewed as a bad practice is that over time each subclass can contain attributes and methods that make no sense for that object the further you go down the chain (in a poorly developed hierarchy). You can end up with a bloated class that contains items that don't make sense for that class.
I'm agnostic about this myself. It seems dogmatic to just say it's bad. Anything is bad when misused.
Is runtime performance. Everytime you invoke a subclass from a superclass you are doing a method call so if you have nested five classes and invoke a method in the base class you will be doing five method invocations. Most compilers/runtimes will try to recognose this and optimise the extra calls away but it is only safe to do this for very simple cases.
Is the sanity of your fellow programmers. If you nest five classes I have to examine all four parent classes to establish you are really calling a method in the base class, it gets tedious. I may also have to step through five method invocations when I debug the program.
"Too much" is a judgement call.
As with most things in programming, sub-classing isn't inherently bad when it makes sense. When the model of your problem solution makes more sense as a hierarchy rather than a composition of parts, subclassing may be the preferred option.
That said, favoring composition over inheritance is a good rule of thumb.
When you subclass, testing can be more difficult (as radarbob noted). In a deep hierarchy, isolating problematic code is more difficult because instantiating a subclass requires bringing in the entire superclass hierarchy and a bunch of additional code. With a compositional approach, you can isolate and test each component of your aggregate object with unit tests, mock objects, etc.
Subclassing can also cause you to expose details of your implementation that you might not want to. This makes it difficult to control access to the underlying representation of your data and it ties you to a particular implementation of your backing model.
One example I can think of is java.util.Properties, which inherits from Hashtable. The Properties class is only intended to store String-String pairs (this is noted in the Javadocs). Because of inheritance, the put and putAll methods from Hashtable have been exposed to users of Properties. While the "right" way to store a property is with setProperty(), there is absolutely nothing preventing a user to call put() and passing in a String and Object.
Basically, the semantics of Properties are poorly defined because of the inheritance. Additionally, should anyone ever want to change the backing object for Properties, the impact will have a much greater impact on code that uses Properties than if Properties had taken a more compositional approach.
Inheritance can break encapsulation in some cases, and if that does happen it is bad. Read for more.
In good old days without inheritance, a library would publish an API and application using it makes use of what is publically visible, and library now has it's own way to handle the internal gears which keep changing without breaking the App.
As the needs evolve, library can evolve and can have more customers; or it can provide extended functionality by inheriting from its own class with other flavors. So far so good.
However, fundamental thing is, if an application take's up the class and begins to subclass it, now a subclass is actually a client rather than an internal partner. However, unlike a clear unambiguous way that public API is defined, the subclass is now much deeper inside the library (access to all private variables and so on). It is not bad yet, but a nasty character can bridge an API which is too much inside the APP as well as in the library-
In essence while this may not always be the case but,
It is easy to misuse inheritance to break encapsulation
Allowing subclassing can be wrong if that point.
Short Quick Answer:
Its depends on what you are doing.
Extended Boring Answer:
A developer can make an app. using either, subclassing or (prototypal) templates. Sometimes one technique works better. Sometimes the other techinque works better.
Choosing with technique works best, also requires, to consider if a "Multiple Inheritance" scenario its present. Even if the programming language only supports single inheritance, templates or prototypes.
Some answers relate to the "multiple inheritance". Altought, not the main question, it's related to the subject. There are several, similar, posts about "multiple inheritance v.s. composition". I had a case where I mix both.