receiveFromAwsS3
Source processor that polls an AWS S3 bucket according to the specified poll configuration.
All objects found in the bucket are processed, unless they match an optional prefix
. Objects selected for processing will be deleted after being processed or optionally moved to a backup bucket.
Properties
Name | Summary |
---|---|
|
Adds a property from the object’s metadata to a list to convert into message exchange properties. If an exchange property with the same name already exists, it will be replaced. |
|
The storage bucket name the object(s) will be read from. Required. |
|
A secret key that the server uses to look up the credentials needed to perform the AWS authentication. Required. |
|
Base URL of the AWS S3 endpoint. Optional. |
|
Region name of the bucket. Optional. |
|
Optional, maximum number of retry attempts the S3 client can perform before failing the request. This option only affects the producer and not the regular flow redelivery mechanism. Defaults to |
|
The delay before performing the first retry after the original request has failed. Optional. |
|
Optional, maximum time to wait between retries. An exponential mechanism is used to calculate the next delay between retries. |
|
The fixed rate poll interval (in milliseconds) to use when polling the AWS S3 bucket. AWS S3 will be polled immediately on startup and then precisely at the rate specified. This, |
|
The fixed delay poll interval (in milliseconds) to use when polling the AWS S3 bucket. AWS S3 will be polled immediately on startup and then with the given delay added after each processing has completed. This, |
|
The poll interval (written as a Quartz Cron Trigger) to use when polling the AWS S3 bucket. The polling will happen at the specified times described by the cron expression. This, |
|
Max objects to return per bucket listing command. If more than |
|
The object key prefix used when processing objects in the bucket. Optional. If not set, all objects in the bucket will be processed. |
|
The bucket name to be used for backing up processed objects. Optional. If not set, the processed object will not be moved (only deleted). |
|
The timeout of the HTTP client socket connection. Optional. |
|
The socket timeout to wait for the first byte of response from the server. Optional. |
|
Optional, descriptive name for the processor. |
|
Required identifier of the processor, unique across all processors within the flow. Must be between 3 and 30 characters long; contain only lower and uppercase alphabetical characters (a-z and A-Z), numbers, dashes ("-"), and underscores ("_"); and start with an alphabetical character. In other words, it adheres to the regex pattern |
|
Optional set of custom properties in a simple jdk-format, that are added to the message exchange properties before processing the incoming payload. Any existing properties with the same name will be replaced by properties defined here. |
|
Whether the incoming payload is available for error processing on failure. Defaults to |
Sub-builders
Name | Summary |
---|---|
Strategy for describing the external system integration. Optional. |
|
Strategy for describing how a processor’s message is logged on the server. |
|
Strategy for archiving payloads. |
|
Strategy that customizes the conversion of an incoming payload by a processor (e.g., string to object). Should be used when the processor’s default conversion logic cannot be used. |
Details
Authentication
The authenticationConfigKey
property supports secrets of type AwsCredentials.
See the Secret Types documentation for formatting details.
Exchange Properties
In addition to any message exchange properties defined by the forwardProperty()
function, the payload always includes the following exchange properties:
Property | Description |
---|---|
|
The name that was assign to the object. |
|
The entity tag (i.e., a hash of the object). The ETag only reflects changes to the contents of the object, not its metadata. |
|
The owner of the object. |
|
The storage class used to store the object. |
|
The creation date of the object in the format |
|
The size of the payload as it was received by the endpoint. |
|
A textual description of the payload, typically used for logging. |
Refer to the AWS S3 documentation for more information on the aws_s3_*
properties.