Processors

Below is a list of every processor supported by the flow-server. Select a processor name to learn more about how that processor can be configured. Processor properties are implemented in the following manner:

processorName {
    propertyName = "value"
    functionName("value")

    subBuilderName {
        propertyName = "value"
    }
}
The descriptions within these pages are also available directly in your IDE. If you are using IntelliJ IDEA, and the "Show quick documentation on hover" setting is checked, you can hover over any processor or property name to see its description.

Amazon Web Services

Name Summary

writeToAwsS3

Processor that writes payloads to an AWS S3 bucket.

lookupFromAwsS3

Processor that looks up an object from an AWS S3 bucket.

receiveFromAwsS3

Source processor that polls an AWS S3 bucket according to the specified poll configuration.

Archiving and Resubmission

Name Summary

archiveLookup

Source processor that exposes a REST endpoint specifically for retrieving archived payloads.

resubmit

Source processor that exposes a REST endpoint specifically for resubmitting archived payloads.

Arrow

Name Summary

writeToArrow

Processor for writing data in the Arrow format using Arrow Flight RPC protocol.

Azure

Name Summary

writeToAzureBlobStorage

Processor that writes payloads to an Azure Blob Storage container.

lookupFromAzureBlobStorage

Processor that looks up an object from an Azure Blob Storage container.

sendToEventHubs

Processor that acts as an Event Hubs producer to publish data to a given Azure Event Hubs instance.

Copying

Name Summary

incrementCounter

Processor that increments the value of a uniquely identified counter set as an exchange property.

setExchangeProperty

Processor that saves a given source to a message exchange property. Value is saved as a string by a given MarshallingFormat.

setPayload

Processor that sets a given source as the new payload.

saveToStash

Processor that saves the given source to a message exchange scoped storage. Stash can then be accessed later on using the given key.

deleteFromStash

Processor that deletes the object stored in the current flow’s stash by the given key.

Cryptography

Name Summary

hash

Processor for converting incoming payload to binary message digest, using the specified hashing algorithm. Non-binary payloads will be converted to binary before being processed.

loadCertificate

Processor for loading a certificate from a secret and set the certificate as the payload.

sign

Processor for generating a digital signature based on asymmetric key cryptography.

verifySignature

Processor for verifying that a message corresponds to a cryptographic signature, ensuring that the message has not been modified in transit. The signing and verification process is based on asymmetric key encryption.

Databases

Name Summary

dbStatement

Processor that executes a statement against a database.

Files

Name Summary

readFiles

Source processor that consumes files over different protocols.

writeFiles

Processor that produces files over different protocols.

Flat Files

Name Summary

parseCsv

Processor that parses a flat file input with data columns separated by a delimiter. This processor is highly configurable but will require almost no configuration to process regular CSV files.

parseExcel

Processor that parses an Excel spreadsheet file (i.e., XLSX and XLS files). This processor is highly configurable but will require almost no configuration to process regular Excel files.

parseHierarchicalCsv

Processor that, like the parseCsv processor, parses flat file input with data columns separated by a delimiter.

Google Cloud

Name Summary

consumeGooglePubSub

Processor that consumes messages from a Google Cloud Pub/Sub subscription.

sendToGooglePubSub

Processor that acts as a Google Cloud Pub/Sub producer to publish messages to a Pub/Sub topic.

writeToGoogleStorage

Processor that writes payloads to a Google Cloud Storage bucket.

lookupFromGoogleStorage

Processor that looks up an object from a Google Cloud Storage bucket.

Internet

Name Summary

restApi

Source processor for exposing REST endpoints. Can be built from an API specification or exposed as a generic endpoint.

restRequest

Processor that produces a message from a REST endpoint.

soapApi

Source processor for exposing SOAP endpoints. Reference to a WSDL specification must be provided.

soapRequest

Processor that acts as a producer of messages to a SOAP endpoint. Reference to a WSDL specification must be provided.

JMS

Name Summary

receiveFromJms

Source processor that consumes messages from a JMS broker.

sendToJms

Processor that acts as a JMS producer to publish data to a given JMS server.

Kafka

Name Summary

sendToKafka

Processor that acts as a Kafka producer to publish key-value pairs to a given Kafka server.

receiveFromKafka

Source processor that consumes messages published to the specified Kafka cluster and topic. The processor produces payloads by reading the record’s value as binary data. To convert the binary data to e.g. json, use an inboundTransformationStrategy in a downstream processor.

MQTT

Name Summary

receiveFromMqtt

Receive messages from a MQTT broker.

sendToMqtt

Send messages to a MQTT broker.

OPC UA

Name Summary

receiveFromOpcUa

Source processor that reads tags (value and status) from an OPC UA server.

RabbitMQ

Name Summary

receiveFromRabbitMq

Source processor that consumes messages from a RabbitMQ queue.

sendToRabbitMq

Processor that acts as a RabbitMQ producer to publish data to a given RabbitMQ broker.

Routing and Scheduling

Name Summary

aggregate

Processor that aggregates fragments based on an identifier (correlation ID) and a given number of expected fragments.

distribute

Processor that distributes messages to a set of flows. The distribution will not be considered complete until all destinations have confirmed reception of the message.

filter

Processor that determines if message payloads may proceed, provided the expression returns true when evaluated against the message payload.

handoff

Source processor that receives messages handed over from other flows.

schedule

Source processor that triggers an event message at configurable times.

scheduledHandoff

Source processor that delays the processing of messages handed over from other flows.

split

Processor that splits the payload into fragments and sends each fragment as a separate message to the given flow.

suspendMessage

Processor that suspends processing of a message and waits for a correlated resume message before continuing.

resumeMessage

Processor that produces a resume message suitable for forwarding to suspendMessage processors to resume processing of suspended messages.

windowedAggregationHandoff

Source processor that aggregates fragment messages based on a period of time or a number of messages, whichever comes first.

SCADA

Name Summary

receiveFromHabitat

Receive data from Habitat, which is a SCADA application from GE Vernova that contains real-time data.

Transformations

Name Summary

compress

Processor for compressing the incoming payload (i.e., by creating a zip archive).

decompress

Processor for decompressing the incoming payload.

fromBase64

Processor that converts a Base64 encoded string to binary. Non-string payloads will be converted to string before being processed.

group

Processor that groups the data in a list based on a grouping key.

map

Processor that transforms the payload using a mapping specification.

toBase64

Processor that converts the incoming payload to a Base64 encoded string. Non-binary payloads will be converted to binary before being processed.

Validation

Name Summary

validate

Processor that validates payloads by a given schema. Fails message processing if validation fails.