Your unit tests are biting you because you are following the traditional approach to testing, as described "by the book". As you can see, many answers are telling you that everything is fine and this is how it is supposed to be. In other words, most people find no fault with this way of unit testing.
I am here to say otherwise.
The traditional approach to testing (unit testing with mocks) is testing against the implementation, not against the interface. Thus, it is white-box testing by nature. As you have already found out, this is very highly problematic and it is bound to bite you.
The solution to this problem is to always test against the interface, not against the implementation. In other words, to practice black-box testing. Mocking frameworks, no matter how nifty and ingenious they are, force white-box testing, therefore they should be avoided like covid-19.
You are not limited to mocks and stubs; you can also use fakes. A fake supports the full functionality of the real thing, (or at any rate, the part of the functionality that we have a use for,) but is much more lightweight than the real thing, and it achieves this by making some severe compromise, such as not persisting anything to disk, which is irrelevant when testing anyway. For example, an in-memory database is a fake that you can use in place of an actual RDBMS.
Once you start making use of fakes, you can start doing integration testing only, as Andres Jaan Tack suggested in his answer. Then, your tests can be black-box tests. Black-box tests are reusable, so if you have two different implementations of the same interface you can use one test to test them both. This comes especially handy if you keep building both real modules and their fakes, because you can test both the fake and the real thing with the same test.
The beautiful thing about a fake is that it is a self-contained module which implements a known abstraction, so it is easy to maintain, whereas every single instance of mocking code scattered throughout the test codebase is an instance of incidental complexity in the tests, and therefore hard to maintain.
For more on this idea, you can read this post:
michael.gr - Incremental Integration Testing
Disclosure: I am the author of that post.