Deployment Verification Tests

The Utilihive SDK provides support for developing flows and testing them on a local test server. This is described in the Functional Tests guide.

In addition to testing flows locally during development, you can also test them after they have been deployed to a real deployment environment. We call this "deployment verification testing". This approach can be used to implement end-to-end tests that verify the behavior and health of your flows after they have been deployed.

Deployment verification tests can be implemented in a similar way to the local functional tests, but the deployment of the flows and its associated resources are managed by integrating with the SDK Deployer. The SDK Deployer will be used to deploy flows and resources to the target environment before the tests are run.

Basic Usage

The basic outline for a verification test class looks like the following:

class MyVerificationFlowTest : ConcurrentTestBase() {
    @Test
    fun `GIVEN x WHEN y THEN z`(ctx: ConcurrentTestContext) {
        ctx.addDeploymentVerificationConfig {
            // configure the deployment
            ...
        }

        deploymentVerification(ctx) {
            // requests and assertions
            ...
        }
    }
}

Note that test classes inherit the ConcurrentTestBase class, and @Test functions are given a ConcurrentTestContext parameter. The ConcurrentTestContext object can be used to configure the deployment process.

The deploymentVerification() function ensures that the configured deployment is performed. Any requests and assertions must be made within the deploymentVerification() callback function block.

If your test depends on the preexistence of a message (e.g., a queued message in a message broker or a file on an SFTP server), the creation of that message should also happen in the flowTest() callback to ensure all flows are properly deployed before the message is consumed.

Test Context

The addDeploymentVerificationConfig builder is used to configure the deployment process. It supports a single, optional configuration property, deploymentProvider, of type DeploymentProvider.

Implementations of the DeploymentProvider interface is responsible for ensuring that the deployment environment is in the desired state before the tests are run.

SdkDeploymentProvider

The SDK provides the SdkDeploymentProvider implementation. It uses the SDK Deployer to deploy flows and resources to the target environment before the tests are executed. It depends on all required SDK Deployer properties being defined.

Since deployment verification is intended to be run automatically, for example in a CI/CD pipeline, the SDK Deployer properties must all be available up front, either in the deployment properties file for the target environment, as system properties or as environment properties. Unlike for the manual SDK Deployer execution, missing properties will not be prompted for, but will cause the deployment to fail. See the SDK Deployer documentation for details on how to set up the configuration.

Default Setup

It is optional to leverage addDeploymentVerificationConfig and setting the deploymentProvider property. If not set, a global instance of SdkDeploymentProvider will be used by default. This single global instance will run the SDK Deployer before the test callbacks are executed.

This is the recommended approach when creating regular deployment verification projects.

When using the default deployment provider, addDeploymentVerificationConfig is not called at all. You just need to implement your test logic within the deploymentVerification() callback function.

class MyVerificationFlowTest : ConcurrentTestBase() {
    @Test
    fun `GIVEN x WHEN y THEN z`(ctx: ConcurrentTestContext) {
        deploymentVerification(ctx) {
            // requests and assertions
            ...
        }
    }
}

Assertions

It is recommended to write assertions with the AssertJ library. The example project already includes AssertJ as a Maven dependency.

For asynchronous tests, such as when testing SFTP file consumption, wrap your assertions in the SDK’s assertOrTimeout() callback function to prevent the test from failing prematurely. For example:

assertOrTimeout(TIMEOUT_DURATION) {
    assertThat(...)
}

Service Account Caveat

There is currently a known issue with this approach.

As of now, the SDK Deployer does not support automatically adding Flow Access to existing Service Accounts when deploying flows. This means that the deployment verification tests will not work for new flows out of the box.

Workaround: After a flow has been deployed for the first time, you must manually add it to a service account in the Connect UI. Then use that service accounts credentials when running the deployment verification tests. There are plans to add Flow Access support to the SDK Deployer. This workaround will not longer be needed when this feature is implemented.

Requests

REST

REST-based flows can be tested by making an HTTP request to the flow endpoint and asserting the response. The SDK provides a restApiEndpoint() function, which returns a WebTarget object. WebTarget methods can then be chained to specify the path, type, etc. For example:

val responseMessage = restApiEndpoint(myFlowSpec)
    .path("api")
    .request()
    .header("Authorization", "Basic $base64Auth")
    .post(json(SimpleValue("input")), SimpleMessage::class.java)

assertThat(responseMessage.message).isEqualTo("output")
The base64Auth value in the example above will be the Base64 encoded credentials of a Basic Authentication Service Account your flow have access to.

The WebTarget is designed to convert a response body into a Java type. Hence the SimpleMessage::class.java statement. The SDK provides the following built-in data classes to help with these response conversions:

Class Description

SimpleValue

Objects with a value property.

SimpleMessage

Objects with a message property.

MessageAcquirementReceipt

Objects with messageId and receiptCreationDateTime properties.

You can also write your own data classes as the need arises.

SOAP

SOAP-based flows can be tested by creating a web service client context and making a request within that context. The client is first configured with the newWebServiceClientConfig builder in the following manner:

val clientConfig = WebServiceClientConfig.newWebServiceClientConfig {
    flowSpec = soapFlowSpec
    wsdlDoc = soapDefinition
    serviceName = "MyService"
    portName = "MyService"
}

The following properties apply to the client configuration:

Property Description

flowSpec

The flow spec to test against. Required.

wsdlDoc

The WSDL document, in string format, for the SOAP service. Required.

serviceName

The name of the service from the WSDL to use in the request. Required.

portName

The name of the port from the WSDL to use in the request. Required.

soapAction

The operation, identified by its soapAction attribute, from the WSDL to use in the request. Optional.

address

The service endpoint address as a URL object. Optional.

username

The username for the request’s authentication. Optional.

password

The password for the request’s authentication. Optional.

After the client configuration has been established, the context is created via the withWebServiceFlowClient() function, as the following demonstrates:

withWebServiceFlowClient(clientConfig) {
    val response = request(...)
    assertThat(response).isEqualTo(...)
}

Calling request() from within this context will make a request to the specified client configuration. The request() function can either accept an XML string or JSON compliant maps and will return the response in the same format.

The following example uses an XML string (and would return an XML string):

val response = request(
    """
    <SayHi xmlns="http://www.bccs.uib.no/EchoService.wsdl">
        <Hi>hello</Hi>
    </SayHi>
    """.trimIndent()
)

The following example uses JSON compliant maps (and would return a map):

val nsMap = mapOf("_default" to "http://www.bccs.uib.no/EchoService.wsdl")
val xmlJson = mapOf(
    "_xmlns" to nsMap,
    "SayHi" to mapOf("Hi" to mapOf("_text" to "hello"))
)

val response = request(xmlJson, nsMap)

See the Payload Types documentation for more help on formatting XML as JSON.

Advanced Usage

Customizing the SdkDeploymentProvider

It is also possible to create custom SdkDeploymentProvider instances and set them as the deployment provider when running tests.

You create a provider instance by calling the newSdkDeploymentProvider builder function. In the callback you can define overrides for any of the SDK Deployer properties.

In this example we override the flowPattern and resourcePattern properties:

val myProvider = newSdkDeploymentProvider {
    flowPattern = "my-flow-.*"
    resourcePattern = "my-resource-.*"
}

You can also configure the provider to not actually perform any deployments. Use this approach if you want to run tests against the environment configured for the SDK Deployer, but without deploying anything new.

val myProvider = newSdkDeploymentProvider(performDeployment = false)

Sharing Custom Deployment Providers

One deployment provider instance will run a single deployment process, no matter how many test contexts they are configured on. They can, and normally should, be shared across multiple tests.

All the tests that rely on the same deployment provider instance will wait until its deployment process is done before the test callback is executed. Each individual deployment provider instance can however run its own deployment process concurrently with other instances. Therefore, it is important to set up each instance to deploy a separate set of flows and resources. Otherwise, the deployment processes will interfere with each other.

val fooProvider = newSdkDeploymentProvider {
    flowPattern = "foo-.*"
}
val barProvider = newSdkDeploymentProvider {
    flowPattern = "bar-.*"
}

@Test
fun `GIVEN foo WHEN y THEN z`(ctx: ConcurrentTestContext) {
    ctx.addDeploymentVerificationConfig {
        deploymentProvider = fooProvider
    }

    deploymentVerification(ctx) {
        // requests and assertions
        ...
    }
}

@Test
fun `GIVEN bar WHEN y THEN z`(ctx: ConcurrentTestContext) {
    ctx.addDeploymentVerificationConfig {
        deploymentProvider = barProvider
    }

    deploymentVerification(ctx) {
        // requests and assertions
        ...
    }
}
When you are using custom deployment providers, you need to set the deploymentProvider on the test context for all your test. If you do not set the deploymentProvider on the test context, the default global instance of SdkDeploymentProvider will be used. This is likely to interfere with your custom deployment providers.

Setting Up a Project With Both Local and Deployment Verification Tests

If you are building with Maven, the verification tests are executed by the Surefire Maven plugin, just like the local functional tests. Those two test types are not intended to be executed together, but during separate phases. You typically want to run the local functional tests while you are developing, using the test phase, and the deployment verification tests in a dedicated build step that are performed in your CI/CD pipeline at deployment time.

One solution would be to have the deployment verification tests in a separate module. This works well for many cases.

Another solution would be to have dedicated project setup for the deployment verification tests, and maintain both the local and deployment verification tests in the same project. Below is an example of how to set up a single Maven project with support for both local and deployment verification tests.

Start by using the *Verification naming pattern for the deployment verification tests, instead of the *Test pattern used by the local tests.

class MyFlowVerification : ConcurrentTestBase() {
    @Test
    fun `GIVEN x WHEN y THEN z`(ctx: ConcurrentTestContext) {
        deploymentVerification(ctx) {
            // requests and assertions
            ...
        }
    }
}

Set up pom.xml with a deployment-verification profile that will execute the verification test during the integration-test phase.

<profile>
    <id>deployment-verification</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <environmentVariables>
                        <DEPLOYMENT_VERIFICATION>true</DEPLOYMENT_VERIFICATION>
                    </environmentVariables>
                    <excludes>
                        <exclude>none</exclude>
                    </excludes>
                    <includes>
                        <include>**/*Verification</include>
                    </includes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

The deployment-verification profile must be selected when running the verification:

mvn integration-test -Pdeployment-verification

The test runners in IDEs like Intellij will typically not adhere to the *Test naming pattern, but will run any @Test annotated function they can find. To ensure that the verification tests are not run by default from the IDE, you can leverage the DEPLOYMENT_VERIFICATION environment variable that the profile sets to true.

Create a utility class that provides a function to skip tests if this environment variable is not set. In this example we are utilizing the assumeTrue function from JUnit.

object VerificationTestUtils {
    const val DEPLOYMENT_VERIFICATION_ENV = "DEPLOYMENT_VERIFICATION"

    fun skipIfNotVerification(testName:String) {
        val verificationTestingActive =
            (System.getenv(DEPLOYMENT_VERIFICATION_ENV) ?: System.getProperty(DEPLOYMENT_VERIFICATION_ENV)) == "true"
        val skipMessage = """Skipping test '${testName}' because '$DEPLOYMENT_VERIFICATION_ENV' is not set to "true"."""
        if (!verificationTestingActive) {
            println(
                """

                $skipMessage

                """.trimIndent()
            )
        }
        assumeTrue(verificationTestingActive, skipMessage)
    }

}

Then make sure to call this function for all deployment verification tests, for example in a @BeforeAll function in the test class.

companion object {
    @BeforeAll
    @JvmStatic
    internal fun skipIfNotVerification() {
        VerificationTestUtils.skipIfNotVerification(requireNotNull(MyFlowVerification::class.simpleName))
    }
}

You will now be able to for example use the IDEs "run all tests" functionality without running the deployment verification tests.

If you want to run the verification test from your IDE you must set the DEPLOYMENT_VERIFICATION environment variable to true in your IDE test execution configuration for the verification test.