IMHO you are asking the wrong question. It does not matter if what you create is called a unit test by some people, or an integration test by others. What matters is here,
is this test useful for your case (so will it help you to avoid certain defects, and will it suffciently reduce the area in code where the root cause of a certain defect may be)?
is it fast enough, even when you are going to run several tests of this kind? (1sec longer does not sound much, but when the original test required 0.001 seconds, and the new one requires 1.001 seconds, running 5000 tests of this kind will make a notable difference)
is it maintainable? Ideally more maintainable than alternative approaches? This depends heavily on the tooling and how good an "in-memory DB" is supported.
If you can answer all of these questions honestly with "yes", then go ahead.
Recommended: The Way of Testivus - which tells you, for example, to follow less dogma (which is a good recommendation not just for testing).