IMHO you are asking the wrong question. It does not matter if what you create is called a unit test by some people, or an integration test by others. What matters is here,
is this test useful for your case (so will it help you to avoid certain defects, and will it suffciently reduce the area in code where you have to search for the root cause forof a certain defect may be)?
is it fast enough, even when you have to are going to run several tests of this kind? (1sec longer does not sound much, but when youthe original test required 0.01001 seconds, and yourthe new one requires 1.01001 seconds, running 5000 tests of this kind will make a notable difference)
is it maintainable? Ideally more maintainable than alternative approaches? This depends heavily on the tooling and how good an "in-memory DB" is supported.
If you can answer all of these questions honestly with "yes", then go ahead.
Recommended: The Way of Testivus - which tells you, for example, to follow less dogma (which is a good recommendation not just for testing).