The purpose of this post is to provide information and answer questions about integration tests. It covers topics such as the importance of integration tests, how to create and set up an XUnit project, how to recreate and seed a database, how to run integration tests in order, and how to mock calls to external APIs so that the integration tests do not rely on the availability of external systems.
Integration Tests
Integration tests are a type of software testing that verifies the interaction and communication between different components in a system. They are designed to test the integration of different parts of a system.
Why integration tests are important?
Integration tests are important because they test how different parts of a system work together and verify that components in the application, such as different classes or modules, work together as expected. This is different from unit tests, which test individual components in isolation.
Integration tests are particularly important when you are building a complex application with many different components that need to work together. By writing integration tests, you can verify that the different parts of the application are communicating and interacting correctly and that the application as a whole is working as intended.
The test pyramid
The test pyramid refers to the idea that, when writing tests, it is generally more efficient and effective to have a larger number of low-level unit tests and a smaller number of high-level end-to-end tests. This is because low-level unit tests are typically easier to write, work with, and maintain than high-level end-to-end tests. In addition, low-level unit tests are usually faster to execute than higher-level intergrain or end-to-end tests, which may require setting up a server or the entire system to run.
Integration tests, which are placed in the middle of the pyramid, cover some of the same areas as unit tests and some of the same areas as end-to-end tests. They are generally used to test the integration of different components or systems covering both the happy and unhappy paths. Unit tests, on the other hand, include tests for niche cases that integration tests may not cover.
As it is mentioned before integration testing brings much more value than other types of testing due to they test the application as a whole without manual interaction.
Integration tests also can be used to test the integration of your application with external dependencies, such as databases or third-party services.
This can help you ensure that your application is functioning correctly in production-like environment.
Overall, integration tests are an important part of the testing process because they help you catch issues that might not be detected by unit tests, and they give you confidence that your application is working as intended.
Types of integration tests
- Big-bang integration test: This approach involves testing all components of a system together, as if they were already integrated. This can be useful for verifying that all components work together as intended, but can be difficult to set up and debug if problems are encountered.
- Mixed integration test: This approach involves testing some components together and others separately, depending on the level of integration that has been achieved. This can be a more flexible approach than the big-bang method, but can still be challenging to set up and debug.
- Riskiest hardware integration test: This approach involves testing the riskiest hardware components first, and then working down to the less risky components. This can be a good approach if hardware components are difficult or costly to replace, as it allows for early identification and resolution of any issues.
- Top-down integration: This approach involves testing the top-level components of a system first, and then working down to the lower-level components. This can be a good approach if the top-level components are well-defined and stable, as it allows for a clear understanding of the overall system before diving into the details.
- Bottom-up integration: This approach involves testing the lower-level components first, and then working up to the higher-level components. This can be a good approach if the lower-level components are well-defined and stable, as it allows for a solid foundation to be built before adding more complex functionality.
It is difficult to say which integration testing approach is the best, as it can depend on the specific requirements and constraints of a project. Different approaches may be more or less suitable for different systems, and the choice of approach may also depend on the preferences and experience of the development team. Some teams may choose to use a combination of different approaches, depending on the needs of the system being tested. In general, it is important to choose an integration testing approach that is appropriate for the specific system being developed, and that allows for the early identification and resolution of any issues that may arise.
The choice of integration testing approach will depend on the specific requirements and constraints of a project. Top-down integration can be simpler to set up, as the top-level components are tested first and any issues can be more easily identified and resolved before moving on to the lower-level components
Creating Integration tests with Xunit
This project will be using WebApplicationFactory which is an in-memory server that allows us to run the web application along the integration tests.
Then a real SQL server database will be executed for running the test application using Docker.
After the database is ready, it will run the entity framework migrations to recreate the database and will seed the tables with the necessary data for each test.
Then it will run a mock server to call third-party-API-dependencies without calling the real APIs.
1.- Create a new project using XUnit
1.- Need to install the next packages
- Microsoft.AspNetCore.Mvc.Testing that will allow us to create a local server to test the application.
- Testcontainers will allow us to create docker containers to run the database in the container.
2.- Create TestFactory class
This class will implement the WebApplicationFactory.
WebApplicationFactory is a class in the Microsoft.AspNetCore.Mvc.Testing namespace that provides a way to test an ASP.NET Core application that is hosted in-memory within the same process as the test. It is used to set up an integration test for an ASP.NET Core application by creating an instance of the application, starting it, and providing an HttpClient instance that can be used to send HTTP requests to the application.
To use WebApplicationFactory, you will need to create a test class that derives from WebApplicationFactory and overrides the ConfigureWebHost method. In this method, you can configure the web host that will be used to start the application, as well as any services that should be added to the application’s service collection. You can then use this test class to create an instance of the application and send HTTP requests to it using the HttpClient instance provided by the WebApplicationFactory.
This is an implementation of a WebApplicationFactory for integration tests in an ASP.NET Core web application. The TestApiFactory class overrides the ConfigureWebHost method to customize the services that are available to the application during the test.
The TestApiFactory class also implements the IAsyncLifetime interface, which defines two methods: InitializeAsync and DisposeAsync. These methods are used to start and stop a test container database (which is an instance of a Docker container running a Microsoft SQL Server database) before and after the integration tests are run.
In the ConfigureWebHost method, the AppDbContext (which is a class that represents the database context for the application) is added to the service collection with a connection string that points to the test container database. The Migrate method is then called on the context to apply any outstanding database migrations, and the Seed.SeedUsers method is called to seed the database with test data.
Here is an example of a test class that uses WebApplicationFactory to test an ASP.NET Core application:
namespace IntegrationsTests.SeedTests
{
public class TestApiFactory : WebApplicationFactory<IApiMarker>, IAsyncLifetime
{
// Test container database instance
private readonly TestcontainerDatabase _dbContainer;
public TestApiFactory()
{
// Build the test container database using the TestcontainersBuilder class
_dbContainer = new TestcontainersBuilder<MsSqlTestcontainer>()
.WithDatabase(new MsSqlTestcontainerConfiguration
{
// Set the password for the test container database
Password = "localdevpassword#123",
// Set the name of the test container database
Database = "integration-db"
})
// Set the image for the test container database
.WithImage("mcr.microsoft.com/mssql/server:2017-latest")
// Set the clean up flag for the test container database
.WithCleanUp(true)
.Build();
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
// Customize the services available to the application during the test
builder.ConfigureTestServices(services =>
{
// Remove the app's ApplicationDbContext registration
var descriptor = services.SingleOrDefault(
d => d.ServiceType ==
typeof(DbContextOptions<AppDbContext>));
if (descriptor != null)
{
services.Remove(descriptor);
}
// Add the AppDbContext to the service collection with a connection string
// that points to the test container database
services.AddDbContext<AppDbContext>(options =>
options.UseSqlServer(
_dbContainer.ConnectionString));
// Get an instance of the AppDbContext from the service provider
var serviceProvider = services.BuildServiceProvider();
using var scope = serviceProvider.CreateScope();
var scopedServices = scope.ServiceProvider;
var context = scopedServices.GetRequiredService<AppDbContext>();
// Apply any outstanding database migrations
context.Database.Migrate();
// Seed the database with test data
Seed.SeedUsers(context);
});
}
// Start the test container database before the integration tests are run
public async Task InitializeAsync()
{
await _dbContainer.StartAsync();
}
// Stop the test container database after the integration tests are run
public async Task DisposeAsync()
{
await _dbContainer.DisposeAsync();
}
}
}
Seed data class:
{
public static class Seed
{
public static async Task SeedUsersAsync(AppDbContext appDbContext)
{
if (await appDbContext.UserEntities.AnyAsync())
{
return;
}
var userData = await File.ReadAllTextAsync("Data/UserSeed.json");
var options = new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true
};
var users = JsonSerializer.Deserialize<List<UserEntity>>(userData, options);
if (users is null)
{
return;
}
await appDbContext.UserEntities.AddRangeAsync(users);
await appDbContext.SaveChangesAsync();
}
}
}
When creating integrations tests order execution might be important.
XUnit does not proportion an easy strategy for order execution. XUnit was created to execute unit tests and unit tests should never depend on the execution of other unit tests.
In this case, it is necessary to order the test cases by implementing the ITestCaseOrderer.
The next example is an implementation of the ITestCaseOrderer interface in the XUnit testing framework. The PriorityOrderer class defines a method called OrderTestCases that takes a sequence of test cases and returns an ordered sequence of test cases.
The test cases are sorted first by their priority, which is determined by the presence and value of a TestPriorityAttribute attribute applied to the test method. Test cases with higher priority values will be run before test cases with lower priority values.
Within each priority group, the test cases are sorted alphabetically by their method name.
The GetOrCreate method is a helper function that gets the value associated with a given key in a dictionary, or creates a new value if the key is not present in the dictionary. It is used to ensure that there is a list of test cases for each priority value in the sorted dictionary.
public class PriorityOrderer: ITestCaseOrderer
{
// Implement the OrderTestCases method of the ITestCaseOrderer interface
public IEnumerable<TTestCase> OrderTestCases<TTestCase>(IEnumerable<TTestCase> testCases) where TTestCase : ITestCase
{
// Create a sorted dictionary to store the test cases by priority
var sortedMethods = new SortedDictionary<int, List<TTestCase>>();
// Iterate over the test cases
foreach (TTestCase testCase in testCases)
{
// Set the default priority to 0
int priority = 0;
// Check if the test method has a TestPriorityAttribute attribute
foreach (IAttributeInfo attr in testCase.TestMethod.Method.GetCustomAttributes(
typeof(TestPriorityAttribute).AssemblyQualifiedName))
{
// If the attribute is present, get the priority value from it
priority = attr.GetNamedArgument<int>("Priority");
}
// Add the test case to the list for its priority level
GetOrCreate(sortedMethods, priority).Add(testCase);
}
// Iterate over the sorted dictionary and yield the test cases in the correct order
foreach (var list in sortedMethods.Keys.Select(priority => sortedMethods[priority]))
{
// Sort the test cases within each priority level alphabetically by method name
list.Sort((x, y) => StringComparer.OrdinalIgnoreCase.Compare(x.TestMethod.Method.Name, y.TestMethod.Method.Name));
foreach (TTestCase testCase in list) yield return testCase;
}
}
// Helper function to get the value for a key in a dictionary, or create a new value if the key is not present
static TValue GetOrCreate<TKey, TValue>(IDictionary<TKey, TValue> dictionary, TKey key)
where TValue : new()
{
TValue result;
// Try to get the value for the key
if (dictionary.TryGetValue(key, out result)) return result;
// If the key is not present, create a new value
result = new TValue();
dictionary[key] = result;
return result;
}
}Next you need to create the Attribute that will be used in the test cases. This attribute class is called TestPriorityAttribute in the XUnit testing framework. The attribute can be applied to a test method to specify a priority for the test.
The AttributeUsage attribute specifies that the TestPriorityAttribute attribute can only be applied to methods, and that it cannot be applied multiple times to the same method.
The TestPriorityAttribute class has a single public constructor that takes an int value representing the priority of the test. It also has a public Priority property that can be used to get the value of the priority.
When the PriorityOrderer class (which I commented on earlier) is used to order the test cases, it will use the priority values specified by the TestPriorityAttribute attributes to determine the order in which the tests should be run.
// Attribute to specify the priority of a test method
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class TestPriorityAttribute : Attribute
{
// Constructor to set the priority value
public TestPriorityAttribute(int priority)
{
Priority = priority;
}
// Public property to get the priority value
public int Priority { get; private set; }
}Now we proceed to crate the Integration Tests
This is a test class called UserControllerTest in the XUnit testing framework. The class contains three test methods that each make a request to the User endpoint of a web API and verify that the response contains a specific user.
The CollectionDefinition attribute specifies that this class is a member of a test collection, which is a logical grouping of tests.
The TestCaseOrderer attribute specifies that the test cases in this class should be ordered using the PriorityOrderer class, which is a custom implementation of the ITestCaseOrderer interface. The PriorityOrderer class will order the test cases based on the values of the TestPriorityAttribute attributes applied to the test methods.
The IClassFixture<TestApiFactory> generic interface specifies that the test class requires a test fixture of type TestApiFactory, which is a class that provides a web API client to the tests. The TestApiFactory fixture is passed to the test class constructor and is used to create an instance of the HttpClient class for each test.
Each test method is decorated with the Fact attribute, which indicates that it is a test method in XUnit. The TestPriorityAttribute attribute is also applied to each test method to specify its priority. Test methods with higher priority values will be run before test methods with lower priority values.
The test methods make a GET request to the User endpoint of the web API using the HttpClient instance, and then deserialize the response into a list of UserEntity objects. Finally, they use the Should().ContainEquivalentOf method of the FluentAssertions library to verify that the list contains an equivalent user to the expected user.
using System.Collections.Generic;
using System.Net.Http.Json;
using System.Threading.Tasks;
using Core.Entities;
using FluentAssertions;
using Xunit;
using Xunit.Abstractions;
namespace IntegrationsTests.SeedTests
{
[CollectionDefinition("MyTestCollection")]
//[TestCaseOrderer(typeof(PriorityOrderer).FullName, typeof(PriorityOrderer).Assembly.GetName().Name)]
[TestCaseOrderer("IntegrationsTests.SeedTests.PriorityOrderer",
"IntegrationsTests.SeedTests, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
public class UserControllerTest : IClassFixture<TestApiFactory>
{
private readonly TestApiFactory _testApiFactory;
private readonly ITestOutputHelper _testOutputHelper;
private static string _test = "";
public UserControllerTest(TestApiFactory testApiFactory, ITestOutputHelper testOutputHelper)
{
_testApiFactory = testApiFactory;
_testOutputHelper = testOutputHelper;
}
[Fact, TestPriority(1)]
public async Task AWhenGetUser_ShouldReturnListOfSeedUsers()
{
_test += "1";
_testOutputHelper.WriteLine(_test);
var expectedUser = new UserEntity
{
Id = 1,
Name = "Lela",
LastName = "Estes"
};
var client = _testApiFactory.CreateClient();
var result = await client.GetAsync("User");
var users = await result.Content.ReadFromJsonAsync<List<UserEntity>>();
//Asserts
users.Should().ContainEquivalentOf(expectedUser);
_test += "2";
_testOutputHelper.WriteLine(_test);
}
[Fact, TestPriority(2)]
public async Task BWhenGetUser_ShouldReturnListOfSeedUsers()
{
_test += "3";
_testOutputHelper.WriteLine(_test);
var expectedUser = new UserEntity
{
Id = 1,
Name = "Lela",
LastName = "Estes"
};
var client = _testApiFactory.CreateClient();
var result = await client.GetAsync("User");
var users = await result.Content.ReadFromJsonAsync<List<UserEntity>>();
//Asserts
users.Should().ContainEquivalentOf(expectedUser);
_test += "4";
_testOutputHelper.WriteLine(_test);
}
[Fact, TestPriority(3)]
public async Task CWhenGetUser_ShouldReturnListOfSeedUsers()
{
_test += "5";
_testOutputHelper.WriteLine(_test);
var expectedUser = new UserEntity
{
Id = 1,
Name = "Lela",
LastName = "Estes"
};
var client = _testApiFactory.CreateClient();
var result = await client.GetAsync("User");
var users = await result.Content.ReadFromJsonAsync<List<UserEntity>>();
//Asserts
users.Should().ContainEquivalentOf(expectedUser);
_test += "6";
_testOutputHelper.WriteLine(_test);
}
}
}The Web API Application
This is the application that is going to be tested.
The UserController class has a single public method called GetUsers, which is decorated with the HttpGet attribute to indicate that it handles GET requests.
The GetUsers method uses the IUserRepository interface to retrieve a list of users from a data store and returns the list in the response as a JSON object. The Ok method is used to create an OkObjectResult object, which represents a successful HTTP 200 response with a body. The OkObjectResult object contains the list of users as its value.
[ApiController]
[Route("[controller]")]
public class UserController : ControllerBase
{
private readonly IUserRepository _userRepository;
public UserController(IUserRepository userRepository)
{
_userRepository = userRepository;
}
[HttpGet]
public async Task<IActionResult> GetUsers()
{
var users = await _userRepository.GetUsersAsync();
return Ok(users);
}
}The IApiMarker is an interface that is used to indicate which application will be used by the WebApplicationFactory in the IntegrationTest project to create the in-memory server to execute the tests. This interface has to live in the WebApplication that is going to be tested.
public interface IApiMarker
{
}Once the integration tests start running you can open the Docker application. You will see the containers initialized.
The database is the one with the name mcr.microsoft.com/mssql/server:2017-latest, the status should be Running and the Port that is a random number that is generated by the Containers library used in the application.
In order to access the database it is necessary to access using the next configuration:
- Host: localhost
- port: 49155 (this port is a random port and will changed every time the test is executed)
- User: sa
- Password: localdevpassword#123 (configured in the TestFactory)
Mock external APIS
Mocking external APIs when doing integration tests is important because it allows you to test the integration of your application with the API without relying on the availability and behavior of the API itself.
There are several benefits to this:
- You can test your application’s integration with the API more reliably and consistently, since you are in control of the API’s behavior and responses.
- You can write tests that simulate different types of responses from the API, such as error responses or responses with different data. This allows you to test how your application handles these scenarios.
- You can test your application’s integration with the API faster, since you don’t have to wait for network requests to the API to complete.
- You can test your application’s integration with the API more safely, since you don’t have to worry about the impact of your tests on the data or behavior of the API.
Overall, mocking external APIs when doing integration tests can help you to test your application’s integration with the API more thoroughly and confidently.
WireMock.Net is an open-source library for .NET that allows you to create mock HTTP servers and APIs in your tests. You can use WireMock.Net to simulate the behavior of an HTTP API in your tests, including returning specific HTTP responses and headers, simulating network delays, and matching requests based on various criteria such as the HTTP method, the URL, or the request body.
WireMock.Net is useful for integration testing, as it allows you to test the integration of your application with an HTTP API without relying on the availability and behavior of the API itself. This can help you to write more reliable, consistent, and fast integration tests, as well as to test your application’s behavior when interacting with different types of APIs.
To use WireMock.Net, you can create an instance of the WireMockServer class and configure it to return specific responses for various HTTP requests. You can then use the HttpClient class or other HTTP libraries to send requests to the mock server and verify the responses. WireMock.Net supports a wide range of features and customization options, and it can be used with any .NET testing framework, such as XUnit, NUnit, or MSTest.
1.-Install WireMock.Net v1.5.13
2.- Create the Class that will contain the Mock server. (MockChiliServer)
3.- The next step is to initialize the MockChiliServer in the TestApiFactory.
4.- The final step is to override inside the ConfigureWebHost the HttpClient with the MockServer URL.
Now, when running the application the system will inject the new configuration and it will call the mock server.
For testing purposes, let’s imagine that we have in our UserController a new method called CAllExternalAPI that will call an external API.
This method will call the next API .
https://chiliapi.com/rest-api/v1/system/apikey?environmentName=test
Now lets write a new test in the UserControllerTest to call this API using wire mock.
When debugging the code we can see that the client is pointing to the mock server URL
Github link to the code:
David Mata blog 