I have a situation where I am running mostly end-to-end tests on my framework:
- I start of with simple fixtures (which are CSV-files).
- These are imported by my application (which I consider to be a black box in my situation).
- When the importing is complete I check with repository classes if they return what I expect to be returned, and in some cases I direct query the database to see if things are as expected.
Now I think code coverage is great, but in this situation (PHP/PHPUnit/XDebug) it makes the executing of my code 50-80% slower (because XDebug is in-between for the code coverage aspect).
So what makes more sense in this case?
- Disable code coverage and have a quicker testsuite that actually tests with real-world scenarios, but without knowing if everything is covered codewise?
- Have a slower test but more insight in what's being tested (and what not?)
To put things in perspective: I now have only somewhat like 6 end-to-end tests which can take up to 12 minutes when code coverage is on, and about 2-3 minutes without code coverage.
What are your thoughts on this?