1

I'm reading a book's chapter ("Controlled Exception Test" in "Testing Object-Oriented Testing") about testing exception handling in a running system (not at unit level). The conclusion is that it is hard to automatically trigger an exception at a given layer in the system other then simulate a cause for an exception.

Wouldn't usage of AspectJ or any other AOP approach solve the problem?

With AOP I can create an aspect that will throw an expected exception and attach it to the method called before exception handling.

3

AOP frameworks normally address cross-cutting concerns. Exception handling is one of them. I may say that I want to log invocation of every method in my business layer and I also want to intercept and log every exception thrown by any method in that business layer.

When it comes to unit testing, I don't want to test that aspect oriented framework intercepts the method call and logs the exception. People who wrote the aspect oriented framework did this testing already and they know that it works.

What you probably want to test is how the code behaves when the exception is thrown. You normally achieve this with mocking frameworks. For example, in my business layer I have a method that returns me latest films:

[LogExceptionThroughAOPFramework]
public IEnumerable<FilmSummary> GetLatestFilms()
{
   var latestFilms = _repository.GetLatestFilms();
   var latestFilmsDTO = Mapper.Map<FilmSummaryDTO>(latestFilms);
   return latestFilmsDTO;
}

Now, above method may fail when it calls _repository.GetLatestFilms();

There is also an imaginary attribute called LogExceptionThroughAOPFramework - it intercepts a method call and logs the exception.

When it comes to testing this method, you don't want to check that AOP framework logs the exception. It's a third party dependency and you don't want to rely on that dependency. Interception has already been tested by people who wrote the framework. If all your exception handling/logging is done by AOP framework, then you don't need to test it. Instead, test logic of LogExceptionThroughAOPFramework and that should be enough.

If you don't use AOP framework, then use a mocking framework. For example:

[TestMethod]
[ExpectedException(typeof(RelevantTypeOfException)]
public void GetLatestFilms_RepositoryThrowsException_ExpectExceptionToBePropagated
{
   var repository = new Mock<ILatestFilmsRepository>();
   repository.Expect(call => call.GetLatestFilms()).Throws(new RelevantTypeOfException());

   var client = new FilmClient(repository);
   // Throws exception
   client.GetLatestFilms();
}

In above test I say that I expect exception to be thrown (attribute above test method). I then configure my repository (mock) to throw an exception when I try to get latest films. Finally, I make a client object and attempt to get a list of latest films. In the constructor I pass a mock repository, which then throws an exception. The test will pass as the exception will be thrown.

Also, you said:

With AOP I can create an aspect that will throw an expected exception and attach it to the method called before exception handling.

If you do that, then what will you be testing? If exception is thrown by aspect, then you won't be testing your business logic. You will be testing aspect throwing exceptions. I don't see any value in that.

Update

You could potentially add an aspect to whatever it is that you are testing.

You would have to add an aspect for each test. Your aspect must be aware of the context its being executed in, i.e. it needs to know that code is executing in a test context.

So if I wanted to test how the system behaves if repository throws an exception, I'd have to write something on the lines of:

[ThrowExceptionOnlyDuringTest("TestScenario")]
[ThrowExceptionOnlyDuringTest("DifferentTestScenario")]
public IEnumerable<FilmSummary> GetLatestFilms()
{
   // Logic to retrieve films
}

I don't know how you would then wire up your test to this aspect. As far as I can see it, this is not much different to setting expectations on mocks. Aspect will create a proxy for method execution and will throw an exception. Mocking framework will do exactly the same, but in a simpler way.

Update 2

I'm assuming that in your case, you are testing integration of the entire system. You have access to all the components and you are trying to find a nice way of making these components fail during integration tests. AOP approach might work here, I'm still not sure.

What happens in the scenario where you don't have control over components that you are integrating with? For example, if your system talks to currency exchange service in another country, there will be no way you'll get it to fail. In this instance you will simply mock this service and make it throw an exception during integration testing. Because of this, I think it makes sense to have an integration tests with a mock that throws an exception in certain scenario.

  • 1
    Thanks. 1) I didn't mean to test at unit level, but at system level, e.g. to see if in case of misconfiguration the threads/process shutdown gracefully, 2) I didn't want to use AOP to log an exception (i.e. act as a part of exception handling), but to throw an exception (i.e. wrap a method that would normally throw this exception). I thought it some test scenario it might be easier induce exceptions with AOP, instead of provoking a system to throw an exception in a natural way. – dzieciou Sep 1 '13 at 21:38
  • Regarding your edit. If you have a complex exception handling, e.g. log an event, notify external systems and shutdown running thread gracefully, triggering an exception with AOP makes sense to me. It might be hard to automate assertion (I may evaluate output manually, e.g. via log reading), but at least I can simulate faulty situation with AOP. – dzieciou Sep 1 '13 at 21:42
  • 1
    Yes, I agree with you on that. In integration testing scenario, if you want something to throw an exception, you'd have to set up an environment for that and try to cause an exception. This is not always feasible though. How do you get an email server connection to fail? You can ask your colleague to pullout network cable as the test runs and hope that he'll do it fast enough :-) If you use AOP to throw an exception, you are fundamentally doing the same thing as the mocking framework (I think)... – CodeART Sep 1 '13 at 21:58
  • 1
    Mock can be a good solution but it might be sometime hard to mock everything. Can you mock filesystem? (Well, you could wrap filesystem with your own facade during app development, and then mock this facade during testing) Can you simulate OutOfMemoryException with mock? – dzieciou Sep 1 '13 at 22:12
  • 1
    The whole point of mocking is to say: when I call this method, I will simulate an OutOfMemoryException. I then want to see what my code does with that exception. I see your point though - how does the rest of the system react if it's truly out of memory. Maybe in those instances the benefits of testing don't outweigh the costs. Think about aircraft engineering. It's not feasible to test how the plane behaves at 10,000ft when it stalls, especially with a pilot onboard. This (youtube.com/watch?v=tm68PK8GCuk) was bad enough. – CodeART Sep 1 '13 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.