map

Processor that transforms the payload using a mapping specification.

See the Data Mapping documentation for help on how to write mapping configurations.

Properties

Name Summary

mapSpec

The mapping specification. Required.

name

Optional, descriptive name for the processor.

id

Required identifier of the processor, unique across all processors within the flow. Must be between 3 and 30 characters long; contain only lower and uppercase alphabetical characters (a-z and A-Z), numbers, dashes ("-"), and underscores ("_"); and start with an alphabetical character. In other words, it adheres to the regex pattern [a-zA-Z][a-zA-Z0-9_-]{2,29}.

exchangeProperties

Optional set of custom properties in a simple jdk-format, that are added to the message exchange properties before processing the incoming payload. Any existing properties with the same name will be replaced by properties defined here.

retainPayloadOnFailure

Whether the incoming payload is available for error processing on failure. Defaults to false.

Sub-builders

Name Summary

messageLoggingStrategy

Strategy for describing how a processor’s message is logged on the server.

payloadArchivingStrategy

Strategy for archiving payloads.

inboundTransformationStrategy

Strategy that customizes the conversion of an incoming payload by a processor (e.g., string to object). Should be used when the processor’s default conversion logic cannot be used.

Details

JSONiq Syntax

Mappings are based on the JSONiq query language, so a lot of what you can do in native JSONiq can also be accomplished here. FLWOR expressions, for instance, are fully supported.

Keep in mind, however, that the $ character in standard JSONiq is used to denote variables. When you write a data mapping as a Kotlin string (i.e., as part of the mapSpec property), you would need to use ${'$'} syntax to insert an actual $ character. Thus, it is recommended to use the mapper’s alias character, #, for JSONiq variables.

For example, a mapping added to the processor might look like the following:

map {
    id = "map-name-from-payload"
    mapSpec = """
        {
          "firstName" : #input.payload.name
        }
    """.trimIndent()
}

See the Data Mapping documentation for more information and examples on writing JSONiq queries.

JSONiq Features

JSONiq borrows heavily from the XQuery query language, meaning the data mapper supports many of the same functions. JSONiq also has several functions of its own. Refer to the JSONiq documentation for a list of JSONiq and XQuery functions that are available to use.

Note that the Utilihive data mapper does not support the following functions that you might find in the JSONiq documentation:

  • collection()

  • compare()

  • decode-from-roundtrip()

  • encode-for-roundtrip()

  • error()

  • escape-html-uri()

  • format-integer()

  • format-number()

  • iri-to-uri()

  • matches() with 3 arguments

  • replace() with 4 arguments

  • tokenize() with 3 arguments

The data mapper also does not support the following native JSONiq features:

  • Setters (e.g., default collation, default ordering mode, etc.)

  • Library modules (i.e., imported modules)

  • External global variables (i.e., variables that are passed a value from the outside environment)

Flow Data

The map processor converts all incoming data to a JSON compliant object available in the mapping as the variable #input. This variable has the following properties:

Property Description

payload

The incoming payload to the processor.

messageId

The unique identifier of the message exchange. Generated by the server.

originalMessageId

The original identifier of the message exchange from when it was initially created. This might differ from the current ID if the message has passed through a processor that generates new exchanges, such as the split processor.

flowId

The ID of the flow the message is currently being processed in.

ownerId

The ID of the organization that owns the flow.

stash

The message exchange’s key/value stash, as set by the saveToStash processor.

messageExchangeProperties

An array of the message’s exchange properties. You can "unbox" the array into a JSONiq sequence to perform search and filter operations on it.

Exchange Properties

If a mapping needs to manipulate the exchange properties, then the returned JSON document must include both a messageExchangeProperties and payload property. In the following example, a new exchange property is added alongside changes to the payload:

let #newExchangeProperties := (#input.messageExchangeProperties[], {
  "name": "newExchangeProperty",
  "value": "newExchangeValue"
})

return {
    "messageExchangeProperties": #newExchangeProperties,
    "payload": {
        "readingQualityRefs": [],
        "readingTypeRef": #input.payload
    }
}

Note that this uses a JSONiq comma separator to concatenate separate values into a single sequence/array.

Number Limits

In the flow-server, JSON compliant objects assume that integers are within the range of Long types and decimals within the range of Double types. In a mapping, values that fall outside of these ranges can still be used as valid numbers. If the number is intended to be part of the mapping’s output, however, then it needs to be converted to a string. Otherwise, an UnsupportedFeature exception is thrown.

For example, the following mapping is able to perform addition on the large integer value but must cast it to a string when formatting the output:

let #maxInteger := 9223372036854775807

return {
  "value" : (#maxInteger + 1) cast as string
}

Select the "Try It" button above to test out the mapping and number logic.

Utilihive Functions

The data mapper comes with the following Utilihive-specific functions:

Function Description

uh:fail()

Throws an IllegalArgumentException, with the error message being the string given to the function. See below for example uses.

uh:capitalized()

Capitalizes the first character of the given string.

uh:decapitalized()

Converts the first character of the given string to lowercase.

uh:uuid4()

Generates a random v4 UUID.

uh:uuid5()

Generates a namespace-based v5 UUID. Must be given name and namespace arguments: uh:uuid5(name, namespace). The name is any random string. The namespace is either a random UUID string or one of these UUID strings pre-defined in UUID RFC. Currently, four namespaces are pre-defined: NameSpace_DNS, NameSpace_URL, NameSpace_OID, and NameSpace_X500.

uh:tableRowLookup()

Looks up a row from a dynamic table in the Resource Registry. See the Dynamic Tables documentation for more details.

uh:deleteTableRows()

Deletes rows from a dynamic table. See the Dynamic Tables documentation for more details.

uh:formattedDateTime()

Maps a given DateTime string from one format to another, converting between time zones as needed. The source, formats, and time zones are set as properties of a config object. See below for an example.

uh:formattedCurrentDateTime()

Creates a DateTime string of the current time based on the given formatting pattern and (optional) time zone. See below for an example.

Fail Function

Because everything in JSONiq is a composable expression, the uh:fail() function can only be used in the context of assigning or returning a value. In the following example, the mapping either throws an exception via uh:fail() or returns a JSON document:

let #firstName := #input.payload.name
let #company := #input.payload.company

return
  if (#company = null or #company = "" or not exists(#company))
  then uh:fail("Company not provided")
  else
  {
    "firstName" : #firstName,
    "company" : #company
  }
{
  "name" : "Anne"
}

In a more complex mapping, the forced failure might take place in a validation/transformation function that is called as part of a FLWOR expression. For example:

let #resolveEmployee := function(#person) {
  {
    "firstName" : #person.name,
    "location" : switch(#person.locationId)
        case 1 return "Norway"
        case 2 return "USA"
        default return uh:fail("Invalid locationId for employee " || #person.name)
  }
}

let #employees := for #person in #input.payload.employees[]
  return #resolveEmployee(#person)

return {
  #input.payload.company : {
    "employees" : [#employees]
  }
}
{
  "company" : "{company-name}",
  "employees": [
    {
      "name" : "Anne",
      "locationId" : 1
    },
    {
      "name" : "Allen"
    }
  ]
}

DateTime Functions

The DateTime functions are used in the following manner:

let #today := uh:formattedCurrentDateTime("yyyy-MM-dd", "CET")

let #prevDate := uh:formattedDateTime({
  sourceDateTime: "2020-02-20 02:04:06",
  sourceFormat: "yyyy-MM-dd HH:mm:ss",
  targetFormat: "dd/MM/yy HH:mm a",
  sourceZone: "CET",
  targetZone: "UTC"
})

return [#today, #prevDate]

In both cases, the time zones are optional. uh:formattedCurrentDateTime() and sourceZone default to UTC. The targetZone property, however, defaults to the value of sourceZone.

Testing

Unit Tests

Unit tests of a map processor use the withMap() function to create a test context where mapping logic is executed against a flow message object. For example:

@Test
fun `GIVEN input THEN data is mapped correctly`() {
    val mappingConfig = mapConfig {
        mapSpec = myMapping
    }
    withMap(mappingConfig) {
        val input = input(...)
        val mappingResult = map(input)
        assertThat(mappingResult).isEqualTo(...)
    }
}

Mappings are written as strings, so the mapping must be converted to a MapConfig object first via the mapConfig builder before it is passed to the withMap() function.

The input() function creates a flow message object with auto-generated values for ownerId, messageId, etc. For example, the statement input(mapOf("value" to "testData")) would result in an object similar to the following:

{
  "flowId": "e6f66ba5-816f-4ae2-946b-e7452c2f82f0",
  "messageId": "ad531fab-3676-49b8-ba31-ff83774e9b61",
  "originalMessageId": "bd92cf06-71cd-4653-805f-7b923f67a86d",
  "ownerId": "292c987d-3960-4f88-ad44-8adccb14ff49",
  "payload": {
    "value": "testData"
  }
}

Stashes and message exchange properties can also be added via the callback function block. For example:

val input = input(mapOf("value" to "testData")) {
    exchangeProperty("key", "value")
    stash("key", "value")
}

The map() function runs the mapping logic from the current context (i.e., the MapConfig object’s mapSpec) and returns the results. Assertions are then written against those results.

Keep in mind that JSON compliant objects and strings are both valid inputs for a mapping. The mapping won’t automatically assume that a JSON string should be parsed as an object. In an actual flow, a JSON string payload can be converted into an object with an inboundTransformationStrategy. When unit testing, however, it is up to you to either recreate the JSON compliant payload with Kotlin’s mapOf() function or use a serialization library to parse a JSON string.

Functional Tests

Functional tests of a map processor are written as part of a flow test similarly as any other processors, for details see Functional Tests.

The tutorial on data mappings provides examples of both unit testing a mapping and functional testing a flow with the mapping.