Unit Testing a class that requests data from multiple sources

Question

Context

I'm working on a project that pulls data from AWS using the various AWS SDKs for .NET. This specific example deals with the AWSSDK.IdentityManagement SDK

The goal is to query information from IAmazonIdentityManagementService and map it to a model that is helpful to the business domain I'm working in

I've been tasked with writing unit tests for the IamService class.

Problem

With the Unit Test setups being so verbose, I can't help but think the method I'm Unit Testing (GetIamSummaryAsync) must be constructed poorly.

I've googled around for things like "Design patterns for mapping multiple data sources to single objects", but the only advice I see given is to use the Adapter or Proxy patterns. I'm not sure how to apply them to this scenario

Question

• Is there a better way I could construct my IamService class to make it easier (more succinct) to test?
• If the Adapter or Proxy patterns are appropriate for this type of scenario, how would they be applied?

public class IamService : IIamService
{
    IAmazonIdentityManagementService _iamClient;

    public IamService(IAmazonIdentityManagementService iamClient)
    {
        _iamClient = iamClient;
    }

    public async Task<IamSummaryModel> GetIamSummaryAsync()
    {
        var getAccountSummaryResponse           = await _iamClient.GetAccountSummaryAsync();
        var listCustomerManagedPoliciesResponse = await _iamClient.ListPoliciesAsync();
        var listGroupsResponse                  = await _iamClient.ListGroupsAsync();
        var listInstanceProfilesResponse        = await _iamClient.ListInstanceProfilesAsync();
        var listRolesResponse                   = await _iamClient.ListRolesAsync();
        var listServerCertificatesResponse      = await _iamClient.ListServerCertificatesAsync();
        var listUsersResponse                   = await _iamClient.ListUsersAsync();

        IamSummaryModel iamSummary = new IamSummaryModel();

        iamSummary.CustomerManagedPolicies.Count = listCustomerManagedPoliciesResponse.Policies.Count;
        iamSummary.CustomerManagedPolicies.DefaultQuota = getAccountSummaryResponse.SummaryMap["PoliciesQuota"];

        iamSummary.Groups.Count = listGroupsResponse.Groups.Count;
        iamSummary.Groups.DefaultQuota = getAccountSummaryResponse.SummaryMap["GroupsQuota"];

        iamSummary.InstanceProfiles.Count = listInstanceProfilesResponse.InstanceProfiles.Count;
        iamSummary.InstanceProfiles.DefaultQuota = getAccountSummaryResponse.SummaryMap["InstanceProfilesQuota"];

        iamSummary.Roles.Count = listRolesResponse.Roles.Count;
        iamSummary.Roles.DefaultQuota = getAccountSummaryResponse.SummaryMap["RolesQuota"];

        iamSummary.ServerCertificates.Count = listServerCertificatesResponse.ServerCertificateMetadataList.Count;
        iamSummary.ServerCertificates.DefaultQuota = getAccountSummaryResponse.SummaryMap["ServerCertificatesQuota"];

        iamSummary.Users.Count = listUsersResponse.Users.Count;
        iamSummary.Users.DefaultQuota = getAccountSummaryResponse.SummaryMap["UsersQuota"];

        return iamSummary;
    }
}

Where the class IamSummaryModel is defined as:

public sealed class IamSummaryModel
{
    public ResourceSummaryModel CustomerManagedPolicies { get; set; } = new ResourceSummaryModel();
    public ResourceSummaryModel Groups { get; set; } = new ResourceSummaryModel();
    public ResourceSummaryModel InstanceProfiles { get; set; } = new ResourceSummaryModel();
    public ResourceSummaryModel Roles { get; set; } = new ResourceSummaryModel();
    public ResourceSummaryModel ServerCertificates { get; set; } = new ResourceSummaryModel();
    public ResourceSummaryModel Users { get; set; } = new ResourceSummaryModel();
}

public sealed class ResourceSummaryModel
{
    public int Count { get; set; }
    public int DefaultQuota { get; set; }
}

The problem I'm facing is that my Unit Tests turn into a mass of code in the Assemble section. I have to mock every call I make to each AWS SDK client method.

Example Unit Test

[Fact]
public async Task GetIamSummaryAsync_CustomerManagerPolicies_MapToModel()
{
    // Arrange
    var iamClientStub = new Mock<IAmazonIdentityManagementService>();
    
    iamClientStub.Setup(iam => iam.ListPoliciesAsync(It.IsAny<CancellationToken>()))
        .Returns(Task.FromResult(
            new ListPoliciesResponse()
            {
                Policies = new List<ManagedPolicy>()
                {
                    new ManagedPolicy(),
                    new ManagedPolicy()
                }
            }
        ));

    // Lots of other mocks, one for each dependency
    
    var sut = new IamService(iamClientStub.Object);

    // Act
    var actual = await sut.GetIamSummaryAsync();

    // Assert
    Assert.Equal(2, actual.CustomerManagedPolicies.Count);
}

Answer 1

There is nothing wrong with this method. It's pulling in a lot of information, but sometimes that's something that has to be done (e.g. for reporting, or preparing a large data transfer).
It's inevitable that when you mock your data source, the more sources you have, the more you have to mock. That isn't easily avoided. However, you can reevaluate your approach that led you here.

1. Does this data need to be combined?

The first question to ask yourself is if combining this data is necessary. If it isn't, and you can keep this data separate, then that's a great way of keeping your codebase simpler and easier to mock (and thus test).
If this data needs to be combined at some point, then refactoring your class just shifts the data-combining logic to another level, where the same unit-testing question now pops up: how to mock in in that layer? Relocating the logic doesn't fix it.

2. Do I need to unit test this?

Secondly, you should question if unit testing is warranted here. Though not everyone agrees (personally, I'm on the fence), there is a reasonable argument to be made for IamService to not be unit tested as it is not a domain logic class, but instead it is a wrapper/mapper of an external resource.

Similarly, I wouldn't test an EntityFramework context class either, unless it contains custom business logic (e.g. automatic auditing fields), because that business logic is in need of testing. The rest of the class is just the implementation of EF, which doesn't warrant testing.

Your IamService is currently devoid of any real business logic, so the argument to not unit test it is quite strong, in my opinion. The argument that the mapping of the IamSummaryModel object counts as business logic is, well, arguable. I don't always test trivial mappings as trivial code should not be tested (note: while I believe this to be correct, I am aware that it's very easy to misuse the "trivial" label on code that is not actually trivial. BEWARE)

3. How do I minimize the effort of mocking?

If you reached this point, you agree that both combining the data and unit testing your class are necessary. That logically concludes in needing to mock all these data sources when testing that class. It has now become an inevitable fact.

But that doesn't mean you can't make your life easier by reusing/simplifying the arrange logic. Let your test class either inherit from a base class used as a fixture, or implement a property containing said fixture. For this answer, I'll choose the inheritance route, but either works.

public class IamServiceTestFixture
{
    protected IamService GetService()
    {
        var mockedAmazonService = GetMockedAmazonService();

        return new IamService(mockedAmazonService);
    }

    private IAmazonIdentityManagementService GetMockedAmazonService()
    {
        var iamClientStub = new Mock<IAmazonIdentityManagementService>();

        // Set up your mocks

        return iamClientStub;
    }
}

public class IamServiceTests : IamServiceTestFixture
{
    [Test]
    public void MyTest()
    {
        // Arrange
        var sut = GetService();

        // Act
        var actual = await sut.GetIamSummaryAsync();

        // Assert
        Assert.Equal(2, actual.CustomerManagedPolicies.Count);
    }
}

This is a very quick implementation of such a fixture. That fixture can do most of the legwork for you. If you have more than one test, which I very much assume you will, this will significantly cut down on the complexity of having to set this up for every individual test.

When setting up the mock, you can rely on values you chose and make accessible via properties, which you can then later reuse for your assertion logic. For example:

public class IamServiceTestFixture
{
    protected ListPoliciesResponse ListPoliciesResponse { get; private set; }

    public IamServiceTestFixture()
    {
         this.ListPoliciesResponse = new ListPoliciesResponse()
         {
             Policies = new List<ManagedPolicy>()
             {
                 new ManagedPolicy(),
                 new ManagedPolicy()
             }
         }
    }

    protected IamService GetService()
    {
        var mockedAmazonService = GetMockedAmazonService();

        return new IamService(mockedAmazonService);
    }

    private IAmazonIdentityManagementService GetMockedAmazonService()
    {
        var iamClientStub = new Mock<IAmazonIdentityManagementService>();

        iamClientStub.Setup(iam => iam.ListPoliciesAsync(It.IsAny<CancellationToken>()))
            .Returns(Task.FromResult(this.ListPoliciesResponse));

        return iamClientStub;
    }
}

public class IamServiceTests : IamServiceTestFixture
{        
    [Test]
    public void MyTest()
    {
        // Arrange
        var sut = GetService();

        // Act
        var actual = await sut.GetIamSummaryAsync();

        // Assert
        Assert.Equal(
            this.ListPoliciesResponse.Policies.Count(), 
            actual.CustomerManagedPolicies.Count()
        );
    }
}

Notice how I set up a specific mocked response, and am then able to use that mocked response to compare it to the actual response received from my unit under test.

If you need to write specific tests for specific policies, then you can add method parameters where necessary, e.g.:

public class IamServiceTestFixture
{
    protected IamService GetService(IEnumerable<ManagedPolicy> policies)
    {
        var mockedAmazonService = GetMockedAmazonService(policies);

        return new IamService(mockedAmazonService);
    }

    private IAmazonIdentityManagementService GetMockedAmazonService(IEnumerable<ManagedPolicy> policies)
    {
        var iamClientStub = new Mock<IAmazonIdentityManagementService>();

        iamClientStub.Setup(iam => iam.ListPoliciesAsync(It.IsAny<CancellationToken>()))
            .Returns(Task.FromResult(new ListPoliciesResponse()
            {
                    Policies = policies
            }));

        return iamClientStub;
    }
}

public class IamServiceTests : IamServiceTestFixture
{
    [Test]
    public void MyTest()
    {
        var customPolicy = new ManagedPolicy();

        // Arrange
        var sut = GetService(new ManagedPolicy[] { customPolicy });

        // Act
        var actual = await sut.GetIamSummaryAsync();

        // Assert
        actual.CustomerManagedPolicies.Should().Contain(customPolicy);
     }
}

You're probably going to have more complex assertion logic when using custom mocked values, but this is just a basic example.

---

Note: as I mentioned in comment to candied_orange's answer, it's advisable to not use interfaces from your libraries in your domain (or at least heavily minimize it), but that's unrelated to the core of your question here so I'm skipping that point.

Answer 2

A class that is aware of data sources cannot be unit tested. It can only be integration tested.

A class that is aware of data structures can be unit tested. What you need is a way to provide the data structures that doesn't require knowledge of the data sources.

These can be everything from hard coded test data to in memory databases. But if you're talking to your real data sources, you aint unit testing.