Thomas Bandt

Über mich | Kontakt | Archiv

ASP.NET Web API - How to make sure all your controllers are covered by integration tests (Or: How to build a spring gun)

Disclaimer: Wording is hard. What is meant by "integation testing" in this context is a set of automated tests that run calls against endpoints of an ASP.NET Web API, no matter what happens behind the scenes regarding data access etc.

By building our API for our latest project we wanted to make sure that at least all positive use-cases are covered by automated tests, so we began to write them. As the API did grow and we were still in the mode of "move fast and break things", it appeared that some endpoints were introduced quickly but were not tested at all.

So I wrote a special test which now runs along with all the real integration tests and ensures that no endpoint is overlooked.

Here's the code:

internal class MakeSureAllEndpointsAreCoveredTest
{
    [Test]
    public void MakeReallyReallySure()
    {
        var testFixtures = Assembly.GetExecutingAssembly()
            .GetTypes()
            .Where(t => t.IsClass && t.IsSubclassOf(typeof(ApiIntegrationTestBase)));

        var testedEndpoints = new HashSet();

        foreach (var testFixture in testFixtures)
        {
            var tests = testFixture
                .GetMethods(BindingFlags.Public | BindingFlags.Instance)
                .Where(t => t.CustomAttributes.Any(a => a.AttributeType == typeof(TestAttribute)));

            foreach (var test in tests)
            {
                var categoryAttribute = test.CustomAttributes.SingleOrDefault(a => a.AttributeType == typeof(CategoryAttribute));

                if (categoryAttribute != null)
                {
                    testedEndpoints.Add(categoryAttribute.ConstructorArguments.First().Value.ToString());
                }
            }
        }

        var controllers = Assembly.GetAssembly(typeof(Controllers.ApiBaseController))
            .GetTypes()
            .Where(t => t.IsClass && t.IsSubclassOf(typeof(Controllers.ApiBaseController)));

        var uncoveredEndpoints = new HashSet();

        foreach (var controller in controllers)
        {
            var methods = controller
                .GetMethods(BindingFlags.Public | BindingFlags.Instance)
                .Where(m => m.ReturnType == typeof(Contracts.UntypedApiResponse)
                    || m.ReturnType.IsGenericType && m.ReturnType.GetGenericTypeDefinition() == typeof(Contracts.ApiResponse<>));

            foreach (var method in methods)
            {
                var endpoint = controller.Name + "." + method.Name + "(" + String.Join("|", method.GetParameters().Select(p => p.Name)) + ")";

                if (!testedEndpoints.Contains(endpoint))
                {
                    uncoveredEndpoints.Add(endpoint);
                }
            }
        }

        var endpointWhiteList = new List
        {
            "CatchAllController.Handle()",
            "InternalToolingController.Get()",
            "VersionController.Get()"
        };

        uncoveredEndpoints.RemoveWhere(endpointWhiteList.Contains);

        if (!uncoveredEndpoints.Any())
            return;

        uncoveredEndpoints.ToList().ForEach(Console.WriteLine);

        throw new AssertionException(uncoveredEndpoints.Count + " endpoint" + (uncoveredEndpoints.Count > 1 ? "s are" : " is") + " not covered by tests. These controllers have been ignored: "
            + Environment.NewLine + "  - "
            + String.Join(Environment.NewLine + "  - ", endpointWhiteList.ToArray()) + "");
    }
}

As we want to keep the Web API project itself clean we did not put the tests inside that project, even if I prefer that method for some other kind of tests (especially unit tests, which I like to be located next to their system under test).

So we first fetch all the tests within the test project/assembly and collect their category attributes. That's the only convention we use.

For a controller like

public class UserFavoritesController : ApiController
{
    public ApiResponse<IEnumerable<UserFavoriteView>>
    {
        // ...
    }
}

the test has to come with a string within the category attribute, that matches the controller name as well as the method name:

public class UserFavoritesController_GET
{
    [Test]
    [Category("UserFavoritesController.Get()")]
    public void Delivers_all_of_the_Users_Favorites()
    {
        // ...
    }
}

Next we are fetching all the controllers and their methods within the API project. By matching them with the signatures collected before we can easily detect if there is a controller or a specific method which isn't covered by a test.

As there are always things you don't need or want to test that way, there is also a possibility to exclude some signatures.

At the end the test will fail if it found any untested endpoint and give you an explicit result, which looks like this:

InvitationsController.Delete(token)

1 endpoint is not covered by tests. These controllers have been ignored:
  - CatchAllController.Handle()
  - InternalToolingController.Get()
  - VersionController.Get()

That really helped us to keep an overview.



« Zurück  |  Weiter »