Software testing techniques are extremely varied, and the more you educate yourself about them, you are going to begin seeing lots of different (and sometimes conflicting) guidance. There is no single 'book' to go by.
I think you are in a situation where you have seen some guidance for unit tests that say things like
- Each test should be stand-alone, and not be affected by other tests
- Each unit test should test one thing, and only one thing
- Unit tests should not hit the database
and so on. And all of those are right, depending on how you define 'unit test'.
I would define a 'unit test' as something like: "a test that exercises one piece of functionality for one unit of code, isolated from other dependent components".
Under that definition, what you are doing (if it requires adding a record to a database before you can run the test) is not a 'unit test' at all, but more of what is commonly called an 'integration test'. (A true unit test, by my definition, won't hit the database, so you won't need to add a record before deleting it.)
An integration test will exercise functionality that uses multiple components (such as a user interface and a database), and the guidance that would apply to unit tests does not necessarily apply to integration tests.
As others have mentioned in their answers, what you are doing is not necessarily wrong even if you do things contrary to some unit test guidance. Instead, try to reason about what you are really testing in each test method, and if you find that you need multiple components to satisfy your test, and some components require pre-configuration, then go ahead and do it.
But most of all, understand that there are many kinds of software tests (unit tests, system tests, integration tests, exploratory tests, etc.), and don't try to apply the guidance of one type to all of the others.