Factories have many advantages which allow for elegant application designs in some situations. One is that you can set the properties of objects you later want to create in one place by creating a factory, and then hand that factory around. But often you don't actually need to do that. In that case using a Factory just adds additional complexity without actually giving you anything in return. Let's take this factory, for example:
WidgetFactory redWidgetFactory = new ColoredWidgetFactory(COLOR_RED);
Widget widget = redWidgetFactory.create();
One alternative to the Factory pattern is the very similar Builder pattern. The main difference is that the properties of the objects created by a Factory are set when the Factory is initialized, while a Builder is initialized with a default state and all properties are set afterwards.
WidgetBuilder widgetBuilder = new WidgetBuilder();
Widget widget = widgetBuilder.create();
But when overengineering is your problem, replacing a Factory with a Builder is likely not much of an improvement.
The most simple replacement for either pattern is of course to create object-instances with a simple constructor with the
Widget widget = new ColoredWidget(COLOR_RED);
Constructors, however, have a crucial drawback in most object-oriented languages: They must return an object of that exact class and can not return a sub-type.
When you need to choose the sub-type at runtime but don't want to resort to creating a whole new Builder or Factory class for that, you can use a factory-method instead. This is a static method of a class which returns a new instances of that class or one of its sub-classes. A Factory which does not maintain any internal state can often be replaced with such a factory-method:
Widget widget = Widget.createColoredWidget(COLOR_RED); // returns an object of class RedColoredWidget
A new feature in Java 8 are method references which allow you to pass methods around, just like you would do with a stateless factory. Conveniently, anything which accepts a method reference also accepts any object which implements the same functional interface, which can also be a full-fledged Factory with internal state, so you can easily introduce factories later, when you see a reason for doing so.