inboundTransformationStrategy

Strategy that customizes the conversion of an incoming payload by a processor (e.g., string to object). Should be used when the processor’s default conversion logic cannot be used.

A strategy converts in one of two ways, and a strategy that mixes them fails when the flow spec is built:

Conversion by format. A from… call names the format the payload arrives in, and a to… call names the format the processor receives, such as fromJsonText().toJsonCompliant(). Each from… function below describes its conversion and options. New flows use this way.
Conversion by properties. Deprecated, and remains for backwards compatibility. It sets the properties characterSet, objectConversionFormat, transformerProperties, schemaId, xmlNamespacePrefixMapping and allowInsecureXmlProcessing. The conversion it performs depends on which properties are set and on the type of the payload that arrives. Each property below names the calls of conversion by format that replace it.

See the Payload Types documentation for more details and examples.

Properties

Name Summary

Name	Summary
`fromBytes()`	Converts bytes to plain text: `fromBytes { characterSet = "ISO-8859-1" }.toPlainText()`. `characterSet` is the character set the bytes are decoded in, and defaults to `UTF-8`. A payload that is already a string passes through unchanged.
`fromPlainText()`	Converts plain text to bytes: `fromPlainText().toBytes { characterSet = "ISO-8859-1" }`. The `characterSet` of `toBytes` is the character set the text is encoded in, and defaults to `UTF-8`. A payload that is already bytes passes through unchanged.
`fromJsonText()`	Converts JSON text, held as bytes or as a string, to a JSON-compliant payload: `fromJsonText { characterSet = "CESU-8" }.toJsonCompliant()`. `characterSet` decodes JSON text that arrives as bytes, and defaults to `UTF-8`. A payload that is already JSON compliant passes through unchanged.
`fromJsonCompliant()`	Converts a JSON-compliant payload to JSON text, such as `fromJsonCompliant().toJsonText { formattedOutput = true }`, to XML text, such as `fromJsonCompliant().toXmlText { characterSet = "ISO-8859-1" }`, or to Avro, such as `fromJsonCompliant().toAvroSingleRecord { schemaId = "AvroSchema:meter-reading:1" }`. The processor receives text as a string and Avro as bytes. `toJsonText { }` takes these options, each optional: `formattedOutput` indents the text. Defaults to `false`. `ndJsonOutput` writes a JSON-compliant list as newline-delimited JSON, one item to a line, and fails any other payload. Defaults to `false`. `characterSet` is the character set a processor encodes the text in when it writes bytes, such as `sendToKafka`. Defaults to `UTF-8`. `toXmlText { }` takes `characterSet` alone, and an XML declaration names it as its encoding. The payload must be a JSON-compliant object that holds XML, as the payload types guide describes. `toAvroRecords { }` encodes a JSON-compliant list, one record for each item, and `toAvroSingleRecord { }` encodes a JSON-compliant object as one record. Each requires `schemaId`, the ID of the Avro schema as `AvroSchema:<resourceId>:<revision>`, and takes no other option.
`fromAvroRecords()`	Converts Avro binary holding a list of records to a JSON-compliant list: `fromAvroRecords { schemaId = "AvroSchema:meter-reading:1" }.toJsonCompliant()`. `schemaId` is required and names the Avro schema as `AvroSchema:<resourceId>:<revision>`. The payload must arrive as bytes. A payload that is already JSON compliant passes through unchanged.
`fromAvroSingleRecord()`	Converts Avro binary holding exactly one record to a JSON-compliant object or list, whichever the record holds: `fromAvroSingleRecord { schemaId = "AvroSchema:meter-reading:1" }.toJsonCompliant()`. `schemaId` is required and names the Avro schema as `AvroSchema:<resourceId>:<revision>`. The payload must arrive as bytes, and binary holding no record or more than one fails the message. A payload that is already JSON compliant passes through unchanged.
`fromHdbArchive()`	Converts a Habitat HDB archive to its row data, a JSON-compliant object that holds the rows of the tables a selection names: `fromHdbArchive { database("BLS") { table("AREA"); table("LEVEL") { fields("ID_") } } }.toJsonCompliant()`. Leaving the selection out, as in `fromHdbArchive().toJsonCompliant()`, converts the whole archive. The selection names databases, tables and fields, and never filters rows: `database(name) { }` converts the database `name`, and every table of it where its block names none. `table(name) { }` converts the table `name`, and every field of it where its block names none. A pseudo-table, such as `RL%`, is converted only when named here. `fields(patterns)` converts the fields matching the patterns, where `` stands for any run of characters. A database or table name takes no ``. Each table keeps its hierarchy columns, whichever fields the selection names: `subscript`, and `parentSubscript` where its rows have a parent. A name given twice, or a `` in a database or table name, fails when the strategy is built. A name the archive does not hold fails the message, listing the names the archive holds. The payload must arrive as bytes, and not as a stream. A payload that is already JSON compliant passes through unchanged. A payload that is not an archive, or a damaged one, fails the message with an error that says what is wrong. The payload types guide describes the row data.
`fromXmlText()`	Converts XML text, held as bytes or as a string, to a JSON-compliant payload: `fromXmlText { schemaId = "XSD:echo-service:1"; namespacePrefix("_default", "urn:echo") }.toJsonCompliant()`. A payload that is already JSON compliant passes through unchanged. `fromXmlText { }` takes these options, each optional: `schemaId` names the XSD or WSDL resources that describe the XML, as `<type>:<resourceId>:<revision>` with `XSD` or `WSDLv1` as the type, separated by commas. `namespacePrefix(prefix, namespace)` sets the prefix of a namespace in the JSON-compliant payload, and the prefix `_default` names the namespace whose elements get none. `allowInsecureProcessing` allows doctype and entity resolution, where the flow-server’s `allowInsecureXmlProcessing` feature toggle allows it too. Defaults to `false`. `characterSet` decodes XML text that arrives as bytes, and defaults to `UTF-8`.
`characterSet`	Deprecated: conversion by properties, which remains for backwards compatibility. Use `fromBytes { characterSet = … }.toPlainText()` to decode bytes to plain text, and `fromPlainText().toBytes { characterSet = … }` to encode plain text to bytes. Use `fromJsonCompliant().toJsonText()` to write an object as JSON text. JSON text and XML text take `characterSet` in the block of their `from…` or `to…` call. Setting this `characterSet` beside a `from…` call fails when the flow spec is built. Optional. Signifies a string transformation using the given character encoding. When transforming from binary to an object, binary data is first transformed to a string. This binary/string decoding defaults to `UTF-8` unless otherwise specified here.
`objectConversionFormat`	Deprecated: conversion by properties, which remains for backwards compatibility. Use the calls of the format instead: `JSON`: `fromJsonText { … }.toJsonCompliant()` or `fromJsonCompliant().toJsonText { … }` `XML`: `fromXmlText { … }.toJsonCompliant()` or `fromJsonCompliant().toXmlText { … }` `AVRO_RECORDS`: `fromAvroRecords { … }.toJsonCompliant()` or `fromJsonCompliant().toAvroRecords { … }` `AVRO_SINGLE_RECORD`: `fromAvroSingleRecord { … }.toJsonCompliant()` or `fromJsonCompliant().toAvroSingleRecord { … }` Setting `objectConversionFormat` beside a `from…` call fails when the flow spec is built. Optional. Used to transform binary or string data to a JSON compliant object (e.g., `MarshallingFormat.JSON`). When working with XML compliant data, marshalling format should be set to `XML`. Both `objectConversionFormat` and `characterSet` are required to transform to an XML string.
`transformerProperties`	Deprecated: conversion by properties, which remains for backwards compatibility. Use `fromJsonCompliant().toJsonText { formattedOutput = true }` for `formattedOutput`, and `fromJsonCompliant().toJsonText { ndJsonOutput = true }` for `ndJsonOutput`. Setting `transformerProperties` beside a `from…` call fails when the flow spec is built. Optional properties to use when transforming to strings. Uses a simple jdk-format for adding property values.
`schemaId`	Deprecated: conversion by properties, which remains for backwards compatibility. Set `schemaId` in the options of `fromXmlText`, `fromAvroRecords`, `fromAvroSingleRecord`, `toAvroRecords` or `toAvroSingleRecord` instead, such as `fromXmlText { schemaId = "XSD:orders:latest" }`. Setting this `schemaId` beside a `from…` call fails when the flow spec is built. Id of a schema for marshalling formats requiring a schema (i.e. Avro, XSD, WSDL). ID must be following the pattern `<type>:<resourceId>:<revision>`. Set `<revision>` to `latest` to retrieve the latest one. If multiple standalone XSD/WSDL resources are to be used, their IDs can be configured as comma separated values. "Standalone" meaning the XSD/WSDL resources do not import/include each other. This property is optional and only relevant for the following conversions: from XML documents to JSON compliant from Avro to JSON compliant and from JSON compliant to Avro When used for XML conversions it will ensure that elements in JSON compliant representations of XML documents are aligned with the requirements specified in the XSD (e.g., correct representation of XML list elements).
`xmlSchemaSpecId`	Deprecated: conversion by properties, which remains for backwards compatibility. Use `fromXmlText { schemaId = … }.toJsonCompliant()` instead. Setting `xmlSchemaSpecId` beside a `from…` call fails when the flow spec is built. ID of a WSDL or XSD specification following the pattern `<type>:<resourceId>:<revision>`. Set `<revision>` to `latest` to retrieve the latest one. If multiple standalone XSD/WSDL resources are to be used, their IDs can be configured as comma separated values. "Standalone" meaning the XSD/WSDL resources do not import/include each other. This property is optional and only relevant for conversions from XML documents to JSON compliant. It can be used to ensure that elements in JSON compliant representations of XML documents are aligned with the requirements specified in the XSD (e.g., correct representation of XML list elements).
`xmlNamespacePrefixMapping`	Deprecated: conversion by properties, which remains for backwards compatibility. Use `fromXmlText { namespacePrefix(prefix, namespace) }.toJsonCompliant()` instead, with one `namespacePrefix` call for each prefix. Setting `xmlNamespacePrefixMapping` beside a `from…` call fails when the flow spec is built. Mapping of prefixes to namespaces in the XML representation of inbound payload. The special `_default` prefix can be assigned to a single namespace, which is then used as the default namespace for the document and not assigned a prefix. This property is optional and only relevant when doing conversions from XML documents to JSON compliant. Defining the prefix mappings here is highly recommended if you are doing any processing of the JSON compliant payload, since it is the only way you can mandate the prefixes yourself. Namespaces not defined here will have automatic prefixes in the form `ns1`, `ns2`, etc. This property is a string on the standard Java Properties format. Each line contains a prefix specification on the format `<prefix> = <namespace>`. Note that namespaces can only be assigned to a single prefix. Duplicated namespaces are not allowed.
`allowInsecureXmlProcessing`	Deprecated: conversion by properties, which remains for backwards compatibility. Use `fromXmlText { allowInsecureProcessing = true }.toJsonCompliant()` instead. Setting `allowInsecureXmlProcessing` beside a `from…` call fails when the flow spec is built. Whether to allow insecure XML features such as doctype / entity resolution. Allowing insecure processing also requires 'allowInsecureXmlProcessing' feature toggle enabled on flow server. Defaults to `false`.

fromBytes()

Converts bytes to plain text: fromBytes { characterSet = "ISO-8859-1" }.toPlainText(). characterSet is the character set the bytes are decoded in, and defaults to UTF-8. A payload that is already a string passes through unchanged.

fromPlainText()

Converts plain text to bytes: fromPlainText().toBytes { characterSet = "ISO-8859-1" }. The characterSet of toBytes is the character set the text is encoded in, and defaults to UTF-8. A payload that is already bytes passes through unchanged.

fromJsonText()

Converts JSON text, held as bytes or as a string, to a JSON-compliant payload: fromJsonText { characterSet = "CESU-8" }.toJsonCompliant(). characterSet decodes JSON text that arrives as bytes, and defaults to UTF-8. A payload that is already JSON compliant passes through unchanged.

fromJsonCompliant()

Converts a JSON-compliant payload to JSON text, such as fromJsonCompliant().toJsonText { formattedOutput = true }, to XML text, such as fromJsonCompliant().toXmlText { characterSet = "ISO-8859-1" }, or to Avro, such as fromJsonCompliant().toAvroSingleRecord { schemaId = "AvroSchema:meter-reading:1" }. The processor receives text as a string and Avro as bytes. toJsonText { } takes these options, each optional:

formattedOutput indents the text. Defaults to false.
ndJsonOutput writes a JSON-compliant list as newline-delimited JSON, one item to a line, and fails any other payload. Defaults to false.
characterSet is the character set a processor encodes the text in when it writes bytes, such as sendToKafka. Defaults to UTF-8.

toXmlText { } takes characterSet alone, and an XML declaration names it as its encoding. The payload must be a JSON-compliant object that holds XML, as the payload types guide describes.

toAvroRecords { } encodes a JSON-compliant list, one record for each item, and toAvroSingleRecord { } encodes a JSON-compliant object as one record. Each requires schemaId, the ID of the Avro schema as AvroSchema:<resourceId>:<revision>, and takes no other option.

fromAvroRecords()

Converts Avro binary holding a list of records to a JSON-compliant list: fromAvroRecords { schemaId = "AvroSchema:meter-reading:1" }.toJsonCompliant(). schemaId is required and names the Avro schema as AvroSchema:<resourceId>:<revision>. The payload must arrive as bytes. A payload that is already JSON compliant passes through unchanged.

fromAvroSingleRecord()

Converts Avro binary holding exactly one record to a JSON-compliant object or list, whichever the record holds: fromAvroSingleRecord { schemaId = "AvroSchema:meter-reading:1" }.toJsonCompliant(). schemaId is required and names the Avro schema as AvroSchema:<resourceId>:<revision>. The payload must arrive as bytes, and binary holding no record or more than one fails the message. A payload that is already JSON compliant passes through unchanged.

fromHdbArchive()

Converts a Habitat HDB archive to its row data, a JSON-compliant object that holds the rows of the tables a selection names: fromHdbArchive { database("BLS") { table("AREA"); table("LEVEL") { fields("ID_*") } } }.toJsonCompliant(). Leaving the selection out, as in fromHdbArchive().toJsonCompliant(), converts the whole archive.

The selection names databases, tables and fields, and never filters rows:

database(name) { } converts the database name, and every table of it where its block names none.
table(name) { } converts the table name, and every field of it where its block names none. A pseudo-table, such as RL%, is converted only when named here.
fields(patterns) converts the fields matching the patterns, where * stands for any run of characters. A database or table name takes no *.
Each table keeps its hierarchy columns, whichever fields the selection names: subscript, and parentSubscript where its rows have a parent.

A name given twice, or a * in a database or table name, fails when the strategy is built. A name the archive does not hold fails the message, listing the names the archive holds.

The payload must arrive as bytes, and not as a stream. A payload that is already JSON compliant passes through unchanged. A payload that is not an archive, or a damaged one, fails the message with an error that says what is wrong. The payload types guide describes the row data.

fromXmlText()

Converts XML text, held as bytes or as a string, to a JSON-compliant payload: fromXmlText { schemaId = "XSD:echo-service:1"; namespacePrefix("_default", "urn:echo") }.toJsonCompliant(). A payload that is already JSON compliant passes through unchanged. fromXmlText { } takes these options, each optional:

schemaId names the XSD or WSDL resources that describe the XML, as <type>:<resourceId>:<revision> with XSD or WSDLv1 as the type, separated by commas.
namespacePrefix(prefix, namespace) sets the prefix of a namespace in the JSON-compliant payload, and the prefix _default names the namespace whose elements get none.
allowInsecureProcessing allows doctype and entity resolution, where the flow-server’s allowInsecureXmlProcessing feature toggle allows it too. Defaults to false.
characterSet decodes XML text that arrives as bytes, and defaults to UTF-8.

characterSet

Deprecated: conversion by properties, which remains for backwards compatibility. Use fromBytes { characterSet = … }.toPlainText() to decode bytes to plain text, and fromPlainText().toBytes { characterSet = … } to encode plain text to bytes. Use fromJsonCompliant().toJsonText() to write an object as JSON text. JSON text and XML text take characterSet in the block of their from… or to… call. Setting this characterSet beside a from… call fails when the flow spec is built.

Optional. Signifies a string transformation using the given character encoding. When transforming from binary to an object, binary data is first transformed to a string. This binary/string decoding defaults to UTF-8 unless otherwise specified here.

objectConversionFormat

Deprecated: conversion by properties, which remains for backwards compatibility. Use the calls of the format instead:

JSON: fromJsonText { … }.toJsonCompliant() or fromJsonCompliant().toJsonText { … }
XML: fromXmlText { … }.toJsonCompliant() or fromJsonCompliant().toXmlText { … }
AVRO_RECORDS: fromAvroRecords { … }.toJsonCompliant() or fromJsonCompliant().toAvroRecords { … }
AVRO_SINGLE_RECORD: fromAvroSingleRecord { … }.toJsonCompliant() or fromJsonCompliant().toAvroSingleRecord { … }

Setting objectConversionFormat beside a from… call fails when the flow spec is built.

Optional. Used to transform binary or string data to a JSON compliant object (e.g., MarshallingFormat.JSON). When working with XML compliant data, marshalling format should be set to XML. Both objectConversionFormat and characterSet are required to transform to an XML string.

transformerProperties

Deprecated: conversion by properties, which remains for backwards compatibility. Use fromJsonCompliant().toJsonText { formattedOutput = true } for formattedOutput, and fromJsonCompliant().toJsonText { ndJsonOutput = true } for ndJsonOutput. Setting transformerProperties beside a from… call fails when the flow spec is built.

Optional properties to use when transforming to strings. Uses a simple jdk-format for adding property values.

schemaId

Deprecated: conversion by properties, which remains for backwards compatibility. Set schemaId in the options of fromXmlText, fromAvroRecords, fromAvroSingleRecord, toAvroRecords or toAvroSingleRecord instead, such as fromXmlText { schemaId = "XSD:orders:latest" }. Setting this schemaId beside a from… call fails when the flow spec is built.

Id of a schema for marshalling formats requiring a schema (i.e. Avro, XSD, WSDL). ID must be following the pattern <type>:<resourceId>:<revision>. Set <revision> to latest to retrieve the latest one. If multiple standalone XSD/WSDL resources are to be used, their IDs can be configured as comma separated values. "Standalone" meaning the XSD/WSDL resources do not import/include each other.

This property is optional and only relevant for the following conversions:

from XML documents to JSON compliant
from Avro to JSON compliant and from JSON compliant to Avro

When used for XML conversions it will ensure that elements in JSON compliant representations of XML documents are aligned with the requirements specified in the XSD (e.g., correct representation of XML list elements).

xmlSchemaSpecId

Deprecated: conversion by properties, which remains for backwards compatibility. Use fromXmlText { schemaId = … }.toJsonCompliant() instead. Setting xmlSchemaSpecId beside a from… call fails when the flow spec is built.

ID of a WSDL or XSD specification following the pattern <type>:<resourceId>:<revision>. Set <revision> to latest to retrieve the latest one. If multiple standalone XSD/WSDL resources are to be used, their IDs can be configured as comma separated values. "Standalone" meaning the XSD/WSDL resources do not import/include each other.

This property is optional and only relevant for conversions from XML documents to JSON compliant. It can be used to ensure that elements in JSON compliant representations of XML documents are aligned with the requirements specified in the XSD (e.g., correct representation of XML list elements).

xmlNamespacePrefixMapping

Deprecated: conversion by properties, which remains for backwards compatibility. Use fromXmlText { namespacePrefix(prefix, namespace) }.toJsonCompliant() instead, with one namespacePrefix call for each prefix. Setting xmlNamespacePrefixMapping beside a from… call fails when the flow spec is built.

Mapping of prefixes to namespaces in the XML representation of inbound payload. The special _default prefix can be assigned to a single namespace, which is then used as the default namespace for the document and not assigned a prefix. This property is optional and only relevant when doing conversions from XML documents to JSON compliant.

Defining the prefix mappings here is highly recommended if you are doing any processing of the JSON compliant payload, since it is the only way you can mandate the prefixes yourself. Namespaces not defined here will have automatic prefixes in the form ns1, ns2, etc. This property is a string on the standard Java Properties format. Each line contains a prefix specification on the format <prefix> = <namespace>. Note that namespaces can only be assigned to a single prefix. Duplicated namespaces are not allowed.

allowInsecureXmlProcessing

Deprecated: conversion by properties, which remains for backwards compatibility. Use fromXmlText { allowInsecureProcessing = true }.toJsonCompliant() instead. Setting allowInsecureXmlProcessing beside a from… call fails when the flow spec is built.

Whether to allow insecure XML features such as doctype / entity resolution.

Allowing insecure processing also requires 'allowInsecureXmlProcessing' feature toggle enabled on flow server.

Defaults to false.

Was this page helpful?