Testing front-end app that hits a REST service

Question

A lot of questions about testing here... And I have another one. Forgive me, but I want to make sure I do this right.

Let's say I have a front-end application, be it a mobile app, or a single-page website, or anything else. This app hits a REST service, that I also develop. The service fetches data from 3rd party REST APIs, processes this data, and returns the results to the application.

My service is well tested and developed in a hexagonal manner, with comprehensive black-box unit tests (canned data) and few integration tests (hitting 3rd party services) to see that adapters are working ok with ports (see here).

My front end is also well tested with unit tests and mock data (it doesn't really hit the service in tests). I feel though that it would be nice if it was tested together with the service I develop. The few things I am unsure of:

1. Should I be using canned data when testing my service, or should I be hitting 3rd party APIs in the tests?
2. Should these be comprehensive automated tests (provided that all functionality is already tested with canned data on front end, and the service is really well tested too, so a lot of these tests would be testing what's already tested), or should it be just few manual tests, like hitting some buttons and seeing if the data comes through? Here I am mainly thinking about the cost/benefit ratio.
3. Are there any frameworks to write comprehensive automated tests like that?

I am sure many of you were in similar situation. So it would be great to get some advice. If you can link a white paper or a conference about the subject, it would be even better. Thanks in advance for any suggestions.

Answer 1

If you do not own the REST API's, your responsibility ends with them

You need tests that check if your application calls the API under your control correctly. Therefore, use the actual application with the actual service (the one you control) but calling mocked services (the ones you don't control).

So...

1. Use canned data for the services you don't control, pipe that data into your actual service and then use that from your application.
2. This should be a very broad test to simply check if calls from your application go through to the service. At this point you should already have thoroughly tested your application with the responses your service might return and you should have thoroughly tested your service with the responses the services you can't control might return. All of this preferably automated. Depending on the amount of releases you do it might be easier to test this manually just prior to release.
3. Depends on the technology used, which might not make this the best website for that question.

Answer 2

1. Yes you should always test the system with all real-world architecture if you can, some problem may be only occurred in real system/data tests (out of memory/locked connection/environment stableness).

2. If the business logic and data communication is proven to be working great, the next thing you should focus on will be stress test for the service: Regards to your server stacks, how many request can it handled in given time? how is the bandwidth/CPU/thread/memory of the web application server you deployed can stand with?

3. For the stress test framework, we tried WAPT and it should be good enough for general stress test case.

Answer 3

In my opinion, there are two distinct scenarios that you are testing.

1) Test that your application is invoking the external APIs properly.

The goal of these tests is to isolate your application code and ensure it is properly invoking and responding to the external APIs. Accordingly, these should be done with mock services and data to eliminate irrelevant noise from the test.

2) Ensure the external APIs are available and responding as expected.

The goal is to ensure that your application has all necessary resources available in order to operate properly. It is not a unit test, since you are not exercising any piece of code. This is a system health monitor. It is generally of very low value to run a single test against an external dependency at compile time, you want these checks to run continuously at periodic intervals and raise proper alerts when they fail. The ideal solution is a separate application that will run along side your application that will monitor all of the external resources your application depends on. Establishing proper monitoring and alerting on these external dependencies through a system health monitor is essential in high availability scenarios in order to respond as rapidly and efficiently as possible to mitigate any service outage.