receiveFromAzureBlobStorage

Source processor that polls an Azure Blob Storage container according to the specified poll configuration.

All blobs found in the container are processed, unless the optional prefix parameter is specified to filter only matching blobs. Blobs selected for processing will be deleted after being processed or optionally moved to a backup container.

The following system-defined exchange properties are set for each processed blob:

Property name Description

azure.blob.name

The name (key) of the blob.

azure.blob.contentType

The content type of the blob.

azure.blob.contentLength

The content length of the blob in bytes.

azure.blob.lastModified

The last modified timestamp of the blob (ISO-8601).

azure.blob.eTag

The ETag of the blob.

azure.blob.containerName

The name of the container from which the blob was read.

User-defined exchange properties:

In addition to the system-defined exchange properties listed above, any user-defined metadata that was set on the blob by the writer (or external system) can be forwarded as exchange properties using the forwardProperty() function. The property name must match the metadata key exactly.

An example of minimum required processor definition:

receiveFromAzureBlobStorage {
  id = "AzureBlob_consumer"
  blobContainerName = "my-container"
  authenticationConfigKey = "azureAuthKey"
  pollFixedDelayMillis = 500
}

Properties

Name Summary

forwardProperty()

Adds a property from the blob’s metadata to a list to convert into message exchange properties. If an exchange property with the same name already exists, it will be replaced.

blobContainerName

The Azure Blob Storage container name to poll for blobs. Required.

endpoint

The Azure Blob Storage endpoint. Optional but must be set if service principal authentication is used. Must not be set if connection string authentication is used.

authenticationConfigKey

A secret key that the server uses to look up the credentials needed to perform the Azure authentication. Required.

pollFixedRateMillis

The fixed rate poll interval (in milliseconds) to use when polling the Azure Blob Storage container. The container will be polled immediately on startup and then precisely at the rate specified. This, pollFixedDelayMillis, or pollCronExpression must be set.

pollFixedDelayMillis

The fixed delay poll interval (in milliseconds) to use when polling the Azure Blob Storage container. The container will be polled immediately on startup and then with the given delay added after each processing has completed. This, pollFixedRateMillis, or pollCronExpression must be set.

pollCronExpression

The poll interval (written as a Quartz Cron Trigger) to use when polling the Azure Blob Storage container. The polling will happen at the specified times described by the cron expression. This, pollFixedRateMillis, or pollFixedDelayMillis must be set.

maxObjectsPerPoll

Max blobs to return per container listing command. If more than maxObjectsPerPoll blobs are available, the consumer will perform a new listing until all blobs are consumed. Azure has an upper (service) limit of 5000. The response might contain fewer items but will never contain more. Optional and defaults to 100.

prefix

The blob name prefix used when processing blobs in the container. Optional. If not set, all blobs in the container will be processed.

moveToContainerName

The container name to be used for backing up processed blobs. Optional. If not set, the processed blob will not be moved (only deleted).

connectionTimeoutMillis

The timeout of the HTTP client socket connection. Optional.

receiveTimeoutMillis

The socket timeout to wait for the first byte of response from the server. Optional.

name

Optional, descriptive name for the processor.

id

Required identifier of the processor, unique across all processors within the flow. Must be between 3 and 30 characters long; contain only lower and uppercase alphabetical characters (a-z and A-Z), numbers, dashes ("-"), and underscores ("_"); and start with an alphabetical character. In other words, it adheres to the regex pattern [a-zA-Z][a-zA-Z0-9_-]{2,29}.

exchangeProperties

Optional set of custom properties in a simple jdk-format, that are added to the message exchange properties before processing the incoming payload. Any existing properties with the same name will be replaced by properties defined here.

Sub-builders

Name Summary

externalSystemDetails

Strategy for describing the external system integration. Optional.

messageLoggingStrategy

Strategy for describing how a processor’s message is logged on the server.

payloadArchivingStrategy

Strategy for archiving payloads.