Well, it's probably worthwhile to mention that there are two patterns here: Factory method pattern and Abstract factory pattern.
Factory method pattern is what you describe. One real class is responsible for creating any and all instances. There is nothing abstract about this. The advantage is that the creation of instances is separate from their usage.
But what happens when the creation of each instance becomes wildly different one to the next and/or your class is becoming cluttered with gigantic if-else chains?
The abstract factory pattern is the next step which addresses this problem. It makes even the creation of each object entirely flexible and potentially in its own class. For example, rather than connect to a database, each with its own parameters and connect string, all concatenated into a single complex factory class, you're creating implementations of an abstract factory, each implementation focused on a single database.
The advantage ultimately remains the same, however you're able to do so without having a overly complicated real factory class. Obviously, if you can do with only the factory method class, use this. No need to implement an abstract factory pattern when the factory method pattern will do fine.
So to answer your question, the reason you'd call the abstract factory pattern abstract is because each factory itself derives from an abstract factory.