My personal perspective from having worked in agile for a few years, is that you really shouldn't fear a large backlog (to a point).
If you are writing self-contained user stories, each of which is effectively complete in and of itself, then you really only need order them by priority and things shouldn't get more "complex" as new stories are added to the list, even if the list is huge.
I find that when people start thinking the backlog is "too big", what they actually tend to mean is that its full of vague, outdated, irrelevant or confusing items, and they've lost the ability to think of it simply as a list of self-contained piece of work ordered by priority, because it's evolved into something else. Something it shouldn't be.
Often this manifests as attempts to organise the backlog into epics or even sortof "category sprints" (often called things like miscellaneous or "needs refinement"), but this only hides the problem, adding even more complexity as the underlying problem with inappropriate things going into the backlog continues, and things hidden in these category sprints get duplicated.
This might not be you, but if you're finding that the above sounds familiar, you might want to consider that the fact you feel you cannot use the backlog for its process indicates that, for some reason, your organisation is not using the backlog for its intended purpose and this is producing confusion and disorder.