I had a bug that was really difficult to track down, because all the unit tests were green, but the production application didn't work properly.
Here's what happened:
I had a filter class that set my application to ignore data that was not in some specified time windows.
- The unit test, which seemed thorough to me, turned green.
- Additionally, my integration tests also produced results as expected.
- Production, however, did not work.
- As a result of the first two bullets, this problem was very difficult to find.
It turned out the problem was that my test dates were using my time zone (America/Chicago) but the production data was providing dates in UTC, which I did not realize, and the logic for the filter wasn't correct for UTC dates. (I was using joda time
Where did my workflow break down?
- Did I fail to produce a spec that specified that the logic needed to handle dates in any time zone?
- Did I fail to thoroughly consider all cases at the unit test level?
- Did I fail to insure the integration test was sufficiently similar to production?
What changes can I make to my workflow to better prevent this sort of mistake in the future?
- How can I more effectively debug a problem when there is an issue in production but not in testing?