Flow Server Configuration Reference
The following is an overview of the configuration properties available for the primary Connect service: flow-server.
This overview is divided into two sections.
The first section details the general server configuration properties, while the second section outlines the configuration properties specific to each Connect plugin component included with the flow-server.
Most of these properties typically do not require modification, as they have default settings that are generally appropriate.
The Connect Helm chart includes the necessary overrides to ensure successful deployment of the flow-server within a GridOS Foundation environment.
If additional overrides are required, they should be added to the Connect Helm chart’s values.yaml file within your environment-specific deployment project, under the following YAML path:
flowserver:
config:
application.yml:
1. Server configuration
blocking-io-dispatcher.executor
Default value: thread-pool-executor
blocking-io-dispatcher.thread-pool-executor.fixed-pool-size
Default value: 32
blocking-io-dispatcher.throughput
Default value: 1
blocking-io-dispatcher.type
Default value: Dispatcher
cluster-dispatcher.executor
Default value: fork-join-executor
cluster-dispatcher.fork-join-executor.parallelism-max
Default value: 4
cluster-dispatcher.fork-join-executor.parallelism-min
Default value: 2
cluster-dispatcher.type
Default value: Dispatcher
datastax-java-driver.advanced.reconnect-on-init
Default value: true
datastax-java-driver.basic.contact-points
Default value: [cassandra:9042]
datastax-java-driver.basic.load-balancing-policy.local-datacenter
Default value: datacenter1
default-flow-processor-dispatcher.attempt-teamwork
Default value: on
default-flow-processor-dispatcher.default-executor.fallback
Default value: fork-join-executor
default-flow-processor-dispatcher.executor
Default value: default-executor
default-flow-processor-dispatcher.fork-join-executor.parallelism-factor
Default value: 3
default-flow-processor-dispatcher.fork-join-executor.parallelism-max
Default value: 64
default-flow-processor-dispatcher.fork-join-executor.parallelism-min
Default value: 8
default-flow-processor-dispatcher.fork-join-executor.task-peeking-mode
Default value: FIFO
default-flow-processor-dispatcher.shutdown-timeout
Default value: 1s
default-flow-processor-dispatcher.throughput
Default value: 5
default-flow-processor-dispatcher.throughput-deadline-time
Default value: 0ms
default-flow-processor-dispatcher.type
Default value: Dispatcher
flow-pipeline-mailbox.mailbox-type
Default value: com.greenbird.metercloud.integration.flowserver.flow.pipeline.FlowPipelineMailbox
flow-processor-mailbox.mailbox-type
Default value: com.greenbird.metercloud.integration.flowserver.flow.processor.FlowProcessorMailbox
flow-source-mailbox.mailbox-type
Default value: com.greenbird.metercloud.integration.flowserver.source.FlowSourceMailbox
logger-mailbox.mailbox-type
Default value: com.greenbird.metercloud.integration.common.pekko.DiscardingLoggerMailbox
management.endpoint.health.show-details
Default value: ALWAYS
management.endpoints.web.exposure.include
Default value: [health, prometheus, flows, saco]
management.health.defaults.enabled
Default value: false
management.health.diskSpace.enabled
Default value: true
management.health.kubernetes.enabled
Default value: true
management.health.livenessState.enabled
Default value: true
management.health.persistenceHealthCheck.enabled
Default value: true
management.health.ping.enabled
Default value: true
management.health.readinessState.enabled
Default value: true
management.health.refresh.enabled
Default value: true
management.prometheus.metrics.export.enabled
Default value: true
management.server.port
Default value: 8090
mc.flow-server.actorSystemInitializationLoggingIntervalMillis
Default value: 10000
mc.flow-server.actorSystemShutdownTimeoutMillis
Default value: 20000
mc.flow-server.archiver.backoff.maxBackOffMillis
Default value: 60000
Maximal backoff time before restarting the actor. See BackoffOpts.onFailure.maxBackoff
mc.flow-server.archiver.backoff.minBackOffMillis
Default value: 5000
Minimal backoff time before restarting the actor. See BackoffOpts.onFailure.minBackoff
mc.flow-server.archiver.backoff.resetPeriodMillis
Default value: 30000
Time without actor failure after which next backoff time will be set to minimal value. See BackoffOnFailureOptions.withAutoReset
mc.flow-server.featureToggles.allowInsecureXmlProcessing
Default value: false
Whether to allow insecure xml processing. This allows backwards compatibility but puts the instance at risk
Recommended to keep it false unless there is a very good reason not to.
mc.flow-server.featureToggles.fileBasedDeployment
Default value: false
Whether to activate the file based flow management component in components-admin
mc.flow-server.featureToggles.httpReverseProxy
Default value: false
mc.flow-server.featureToggles.loadTestMode
Default value: false
Whether to activate internal flow server load test framework and allow load testing
mc.flow-server.featureToggles.localFileAccess
Default value: false
Whether to enable the FILE protocol for readFiles and writeFiles processors
mc.flow-server.featureToggles.localFlowPipelineDeployment
Default value: false
If true: Deploy eligible flows pipelines locally alongside their sources. This removes the remoting
between sources and their flow pipelines where possible. Pipelines containing stateful singleton processors such as
persistent processors will still be deployed as sharded cluster singletons.
If false: All flow pipelines will be deployed as sharded cluster singletons.
mc.flow-server.featureToggles.managementApiForCustomers
Default value: false
mc.flow-server.featureToggles.resourceRegistryIntegration
Default value: false
If true: Integration with Resource Registry will be enabled alongside following features:
* using resources from Registry by flow-server
* Customer API for Resource Registry (facade from flow-server to resource registry)
mc.flow-server.featureToggles.schemaBasedRestApi
Default value: false
Whether or not to support restApi sources with no resourceClass configured
mc.flow-server.featureToggles.sideChannelMetrics
Default value: true
Whether to enable recording of side channel processing metrics
mc.flow-server.flow-entity.flowComponentShutdownTimeoutSeconds
Default value: 10
mc.flow-server.flow-entity.flowShutdownTimeoutSeconds
Default value: 10
The maximum time a flow manager can wait for a flow to shut down
mc.flow-server.flow-entity.sourceDeprovisioning.errorLoggingAttemptThreshold
Default value: 10
mc.flow-server.flow-entity.sourceDeprovisioning.reminderDelayMills
Default value: 15000
mc.flow-server.flow-entity.sourceDeprovisioning.timeoutMillis
Default value: 60000
mc.flow-server.flow-entity.sourceDeprovisioning.warnLoggingAttemptThreshold
Default value: 3
mc.flow-server.flow-entity.sourceProvisioning.errorLoggingAttemptThreshold
Default value: 10
mc.flow-server.flow-entity.sourceProvisioning.reminderDelayMills
Default value: 30000
mc.flow-server.flow-entity.sourceProvisioning.warnLoggingAttemptThreshold
Default value: 3
mc.flow-server.flow-entity.sourceUpdate.errorLoggingAttemptThreshold
Default value: 10
mc.flow-server.flow-entity.sourceUpdate.reminderDelayMills
Default value: 30000
mc.flow-server.flow-entity.sourceUpdate.warnLoggingAttemptThreshold
Default value: 3
mc.flow-server.flow-pipeline.failureNotification.errorLoggingAttemptThreshold
Default value: 5
mc.flow-server.flow-pipeline.failureNotification.reminderDelayMills
Default value: 20000
mc.flow-server.flow-pipeline.failureNotification.warnLoggingAttemptThreshold
Default value: 2
mc.flow-server.flow.concurrencyMaxNoUpperBound
Default value: 100
mc.flow-server.flow.defaultLengthOfPayloadToLog
Default value: 100
mc.flow-server.flow.management.forcedStateRemovalTimeoutSeconds
Default value: 30
Timeout for force-removing flow source persistent state
mc.flow-server.flow.management.getFlowListTimeoutMillis
Default value: 10000
Timeout to query for flow status(eg. list flows)
mc.flow-server.flow.management.regionAskTimeoutMillis
Default value: 20000
Timeout for sending commands to flow
mc.flow-server.flow.reentry.maxNoOfReentries
Default value: 30
Hard limit of allowed reentries (looped messages) to a source.
mc.flow-server.flow.reentry.supportedSourceProcessorConfigs
Default value: Handoff, ScheduledHandoff
Short-names of the source processors that support reentry. Reentry for any other source will be rejected.
mc.flow-server.health.persistenceHealthCheckTimeoutInMillis
Default value: ${mc.flow-server.health.cassandraHealthCheckTimeoutInMillis:2000}
Deprecated in favor of persistenceHealthCheckTimeoutInMillis
cassandraHealthCheckTimeoutInMillis: 2000
mc.flow-server.largeMessages.byteThreshold
Default value: 100000
Set this to a negative number (e.g. -1) to disable side-channeling.
Changing this property must be seen in combination with the following Pekko persistence config properties.
These properties define how many entries there should be in a partition and in the query buffer, while this property
says how large those messages can be.
- pekko.persistence.cassandra.journal.target-partition-size
- pekko.persistence.cassandra.query.max-buffer-size
mc.flow-server.largeMessages.defaultMessageSizeSamplingFrequency
Default value: 1
How often to measure the size of messages by serialization for side channeling.
⇐ 0: Do as few measurements as possible. Will fall back to previous measurements registered in
exchange properties. Risk: If Cassandra persistence is used; Payloads sizes might change considerably
downstream of the measurement, which can lead to large messages being stored to Cassandra and create huge
partitions that can take down Cassandra.
1: The size of the messages will always be checked. Risk: Performance impact.
> 1: The size of every n-th message will be checked. Risk: If the message size is normally small,
a few large messages will trigger excessive side channeling for all the small messages.
We opt for 1 as the default value, since that is the safest option.
mc.flow-server.largeMessages.sideChannel.defaultMessageMillisToLive
Default value: 30000
mc.flow-server.largeMessages.sideChannel.implementationClass
Default value: com.greenbird.metercloud.integration.flowserver.common.sidechannel.MessageSideChannelFake
mc.flow-server.largeMessages.sideChannel.maxDecompressedPayloadSizeBytes
Default value: 1073741824
Maximum decompression size of the side-channeled payload in bytes. If the decompressed payload exceeds this size, message processing will fail.
mc.flow-server.largeMessages.sideChannel.messageTimeToLiveMultiplier
Default value: 2.0
mc.flow-server.largeMessages.sideChannel.payloadCompressionFormat
Default value: NONE
side channel payload can be compressed/decompressed using following formats: ZIP, GZIP, ZSTD
and NONE (default, compression is not performed)
mc.flow-server.largeMessages.sideChannel.payloadCompressionStrategy
Default value: Default
side channel payload compression strategy for zip compression, possible values: Default, BestSize, BestSpeed
mc.flow-server.largeMessages.sideChannel.properties
Default value: `exampleProperty1 = propertyValue1 exampleProperty2 = propertyValue2 `
mc.flow-server.largeMessages.sideChannel.totalRequestTimeoutMillis
Default value: 1000
mc.flow-server.maxNumberOfShards
Default value: 30
Maximum number of shards that. Value is set as Pekko documentation suggests which is
10 * number of nodes
mc.flow-server.metrics.histogram.numberOfSamples
Default value: 10
mc.flow-server.metrics.histogram.sampleExpirySeconds
Default value: 30
mc.flow-server.metrics.processorPattern
Default value: DbStatementConfig|DistributionConfig|FileWriterConfig|GooglePubSubProducerConfig|GoogleStorageWriterConfig|KafkaProducerConfig|RestRequestConfig|SapBapiConfig|SendEmailConfig|SoapRequestConfig|SplitterConfig|SuspendMessageConfig
mc.flow-server.multi-site.askTimeoutInMillis
Default value: 10000
Timeout for 'asking' site state provider (global singleton actor)
mc.flow-server.multi-site.enabled
Default value: false
Flag for switching multi-side mode
mc.flow-server.multi-site.flowReconciliationTimeoutInMillis
Default value: 60000
Timeout for waiting for flows to be reconciled on state change before retrying (GlobalSiteStateActor)
mc.flow-server.multi-site.saco.name
Default value: OVERRIDE_ME
mc.flow-server.multi-site.site-awareness-client.oauth2-authentication.client-id
Default value: OVERRIDE_ME
oauth client id
mc.flow-server.multi-site.site-awareness-client.oauth2-authentication.client-secret
Default value: OVERRIDE_ME
oauth client secret
mc.flow-server.multi-site.site-awareness-client.oauth2-authentication.issuer-host
Default value:
the virtual host of the Authorization Server set as 'Host' header (optional)
e.g. A Zitadel virtual instance runs on one domain and represents one token 'issuer'.
mc.flow-server.multi-site.site-awareness-client.oauth2-authentication.scope
Default value: OVERRIDE_ME
oauth2 scope
mc.flow-server.multi-site.site-awareness-client.oauth2-authentication.tls-certificate-authority-pem
Default value:
trusted certificate authority in PEM format
optional
mc.flow-server.multi-site.site-awareness-client.oauth2-authentication.token-endpoint
Default value:
the url to the root of Authorization Server (should end with /token)
optional - if not set, the client will not use OAuth2 authentication
mc.flow-server.multi-site.site-awareness-client.rest-client.api-root
Default value: OVERRIDE_ME
the url to the root of the Admin REST API (e.g. https://<foundation.server>/site/v1)
mc.flow-server.multi-site.site-awareness-client.rest-client.connect-timeout-millis
Default value: 3000
tcp connection timeout when connecting
mc.flow-server.multi-site.site-awareness-client.rest-client.read-timeout-millis
Default value: 15000
tcp read timeout ("socket timeout") when reading data
mc.flow-server.multi-site.site-awareness-client.rest-client.tls-certificate-authority-pem
Default value:
trusted certificate authority in PEM format
optional
mc.flow-server.multi-site.site-awareness-client.rest-client.tls-ignore-host
Default value: false
whether to skip hostname verification
mc.flow-server.multi-site.site-awareness-client.site-name
Default value: OVERRIDE_ME
name of the site this server deployed to. Do '$ kubectl get site' to get the value
mc.flow-server.processor.defaultLogFullMessageOnProcessing
Default value: false
mc.flow-server.processor.defaultLogMessageDescription
Default value: false
mc.flow-server.processor.defaultMaxStackTraceDepth
Default value: 5
mc.flow-server.processor.handoffActorReferenceTimeoutMillis
Default value: 5000
mc.flow-server.processor.persistent.actorSelectionResolveTimeoutMillis
Default value: 5000
mc.flow-server.processor.persistent.defaultNoOfMessagesBetweenSnapshots
Default value: 1000
mc.flow-server.processor.persistent.maximumMaxAgeInSecondsForMessagesToKeep
Default value: 2592000
30 days; warning: Long.MAX_VALUE in maxAgeInSecondsForMessagesToKeep is reserved and stands for infinity (message never expire)
mc.flow-server.processor.persistent.maximumMaxNoOfMessagesToKeep
Default value: 10000
mc.flow-server.processor.persistent.maximumNoOfMessagesBetweenSnapshots
Default value: 10000
mc.flow-server.processor.persistent.maximumPurgeIntervalInSeconds
Default value: 60000
The interval of purging old messages from memory is normally calculated to be maxAge/3. To avoid that the intervals
become too large we also apply this maximum interval to the calculation: min(maxAge/3, maximumPurgeIntervalInSeconds).
mc.flow-server.processor.persistent.memoryUsageReportIntervalInSeconds
Default value: 1800
mc.flow-server.processor.persistent.memoryUsageWarningThresholdInBytes
Default value: 1000000
mc.flow-server.processor.persistent.minimumMaxAgeInSecondsForMessagesToKeep
Default value: 10
mc.flow-server.processor.persistent.minimumMaxNoOfMessagesToKeep
Default value: 1
mc.flow-server.processor.persistent.minimumNoOfMessagesBetweenSnapshots
Default value: 100
mc.flow-server.processor.persistent.supervisor.backoff.maxBackOffMillis
Default value: 60000
mc.flow-server.processor.persistent.supervisor.backoff.minBackOffMillis
Default value: 5000
mc.flow-server.processor.persistent.supervisor.backoff.resetPeriodMillis
Default value: 30000
mc.flow-server.resource-registry.actuatorApiRoot
Default value: /actuator
Path to the internal resource API root (the root above [resources|secrets]{orgShortName}/…).
mc.flow-server.resource-registry.actuatorPort
Default value: 8090
Resource Registry endpoint exposing internal API
mc.flow-server.resource-registry.connectionTimeoutInMillis
Default value: 20000
Connection timeout used by HTTP Client when communicating with Resource Registry
mc.flow-server.resource-registry.customerApiRoot
Default value: /api/{orgShortName}
Path to the customer resource API root (the root above /resources/{type}/{it}/…). Supports {orgShortName} replacement.
mc.flow-server.resource-registry.customerPort
Default value: 80
Resource Registry endpoint exposing customer API
mc.flow-server.resource-registry.readTimeoutInMillis
Default value: 60000
Read timeout used by HTTP Client when communicating with Resource Registry
mc.flow-server.resource-registry.url
Default value: http://utilihive-integration-resource-registry
URL for Resource Registry
mc.flow-server.security.authConfig.flow-server-poller.userName
Default value: flow-server-poller
mc.flow-server.security.authorizationSyncCacheService.connectionTimeoutInMillis
Default value: 20000
mc.flow-server.security.authorizationSyncCacheService.flowAccessRequestUrl
mc.flow-server.security.authorizationSyncCacheService.readTimeoutInMillis
Default value: 60000
mc.flow-server.security.authorizationSyncCacheService.retryDelayInMillis
Default value: 10000
mc.flow-server.security.authorizationSyncCacheService.timeToLiveInMillis
Default value: 60000
mc.flow-server.security.payloadLoggingOwnersAllowed
Default value: (?!)
mc.flow-server.source.actor.defaultLogFullDeadLetterMessages
Default value: false
mc.flow-server.source.actor.defaultLogFullMessageOnDelivery
Default value: false
mc.flow-server.source.actor.defaultLogMessageDescription
Default value: true
mc.flow-server.source.actor.defaultMaxBufferSizeOneWay
Default value: 5
mc.flow-server.source.actor.defaultMaxBufferSizeRequestResponse
Default value: 5
mc.flow-server.source.actor.defaultMaxNumberOfUnconfirmedMessages
Default value: 5
mc.flow-server.source.actor.defaultMaxStashSizeOneWay
Default value: 100000
mc.flow-server.source.actor.defaultMaxStashSizeRequestResponse
Default value: 0
mc.flow-server.source.actor.defaultRedeliveryMaxNo
Default value: 5
mc.flow-server.source.actor.defaultRedeliveryMillis
Default value: 15000
mc.flow-server.source.actor.defaultSourcePurgeCheckIntervalMillis
Default value: 1000
mc.flow-server.source.actor.deliveryCounterInFlightMessageFactor
Default value: 2
Factor that will be used to scale maximum number of messages that will be stored in DeliveryCounterActor.
Initial number of maximum messages is based on maximum inflight messages that corresponding FlowSourceActor has configured.
mc.flow-server.source.actor.endpoint.creationRetryDelayMillis
Default value: 10000
Delay before retrying endpoint-creation after encountering a transient error
mc.flow-server.source.actor.endpoint.creationRetryDelayMultiplier
Default value: 2
Multiplier to apply to the retry-delay between endpoint creation retry attempts
mc.flow-server.source.actor.endpoint.maxCreationRetryDelayMillis
Default value: 60000
Maximum delay between endpoint creation retry attempts
mc.flow-server.source.actor.lastRedeliveryExhaustionFactor
Default value: 0.8
In order to compensate the limitation of Pekko’s at-least-once delivery on not supporting 0 redeliveries, we multiply the
configured redeliveryMillis with this factor to exhaust the (last) redelivery attempt before the redeliveryMillis is reached.
This approach is suitable for cases where redeliveryMillis is configured smaller than e.g., lastRedeliveryReductionThresholdMillis
mc.flow-server.source.actor.lastRedeliveryReductionMillis
Default value: 1000
If redeliveryMillis is configured larger than e.g., lastRedeliveryReductionThresholdMillis, we will reduce the redeliveryMillis
with this constant value instead of using lastRedeliveryExhaustionFactor, to avoid a bigger discrepancy between the configured
redeliveryMillis and the real (last) redelivery time.
This constant value is inferred from lastRedeliveryReductionThresholdMillis and the lastRedeliveryExhaustionFactor:
lastRedeliveryReductionThresholdMillis * (1 - lastRedeliveryExhaustionFactor) = 5000 * (1 - 0.8) = 1000
mc.flow-server.source.actor.lastRedeliveryReductionThresholdMillis
Default value: 5000
Threshold used for deciding when to apply lastRedeliveryExhaustionFactor and when to use lastRedeliveryReductionMillis
mc.flow-server.source.actor.maximumShutdownRetryToEscalateToWarning
Default value: 30
mc.flow-server.source.actor.minimumSnapshotTriggeringMessageNo
Default value: 1000
Minimum messages received before a snapshot can be created
mc.flow-server.source.actor.missingSnapshotWarningThresholdMillis
Default value: 604800000
Currently, for flow source actor, snapshotting is only triggered when the actor is idle.
If minimumSnapshotTriggeringMessageNo has passed and the actor hasn’t become idle after the configured threshold, indicating the source is overloaded.
mc.flow-server.source.actor.processingTimeoutBufferMillis
Default value: 5000
mc.flow-server.source.actor.repeatedMissingSnapshotWarningDelayMillis
Default value: 86400000
If a flow source is overloaded, we log errors with the configured interval.
mc.flow-server.source.actor.snapshotCheckPeriodMillis
Default value: 600000
How often to check if a snapshot should be created. Snapshot is only created if the actor is idle and there are messages that have been processed since the last snapshot.
mc.flow-server.source.actor.snapshotMessageDeletionBatchSize
Default value: -1
Maximum number of messages to delete at the time (i.e. pr. batch) after a successful snapshot. This should be set
for environments where the persistence-plugin struggles when deleting large amounts of data (e.g. jdbc).
Use zero or a negative value to turn off batching.
mc.flow-server.source.manager.global.localManagerDeathWatchDelayMillis
Default value: 3000
UINT-2525: Local manager death watch delay to avoid race condition between the local manager
registration and the new cluster member being registered in the provisioner nodes cluster member list.
mc.flow-server.source.manager.global.singletonProxyBufferSize
Default value: 1000
Number of messages to buffer in the proxy to the singleton-proxy, in case the actual singleton is unavailable (eg. moving/starting etc.)
see https://pekko.apache.org/docs/pekko/current/cluster-singleton.html
mc.flow-server.source.manager.global.snapshotTriggeringMessageNo
Default value: 1000
mc.flow-server.source.manager.global.sourceDeploymentRetryAttemptsErrorThreshold
Default value: 10
mc.flow-server.source.manager.global.sourceDeploymentRetryAttemptsWarnThreshold
Default value: 3
mc.flow-server.source.manager.global.sourceDeploymentRetryIntervalMillis
Default value: 30000
mc.flow-server.source.manager.global.sourcePersistenceIdReuseDelayMillis
Default value: 10000
Delay before re-using persistenceId (i.e. deploying purging sources, re-deploying singleton sources) to
avoid conflicting writes in non-deterministic conditions where timing is used to make decisions (e.g. split-brain).
mc.flow-server.source.manager.global.sourceUndeploymentRetryAttemptsErrorThreshold
Default value: 10
mc.flow-server.source.manager.global.sourceUndeploymentRetryAttemptsWarnThreshold
Default value: 3
mc.flow-server.source.manager.global.sourceUndeploymentRetryIntervalMillis
Default value: 30000
mc.flow-server.source.manager.global.supervisor.backoff.maxBackOffMillis
Default value: 60000
mc.flow-server.source.manager.global.supervisor.backoff.minBackOffMillis
Default value: 5000
mc.flow-server.source.manager.global.supervisor.backoff.resetPeriodMillis
Default value: 30000
mc.flow-server.source.manager.global.verifyInitialLocalDeploymentRequestsAreHandledDelayMillis
Default value: 10000
mc.flow-server.source.manager.global.verifyProvisioningRequestsOnStartupDelayMillis
Default value: 30000
When starting, after this delay the global provisioner will verify that all provisioning requests have been processed.
Having this on a delay will allow the provisioner to handle registration and deregistration of local managers, to
avoid unnecessary overhead.
mc.flow-server.source.manager.local.purgeOrphanedMetricsIntervalSeconds
Default value: 86400
mc.flow-server.source.manager.local.registration.errorLoggingAttemptThreshold
Default value: 20
mc.flow-server.source.manager.local.registration.reminderDelayMills
Default value: 30000
mc.flow-server.source.manager.local.registration.warnLoggingAttemptThreshold
Default value: 5
mc.flow-server.source.manager.local.supervision.backoff.maxBackOffMillis
Default value: 60000
mc.flow-server.source.manager.local.supervision.backoff.minBackOffMillis
Default value: 1000
mc.flow-server.source.manager.local.supervision.backoff.randomFactor
Default value: 0.2
mc.flow-server.source.manager.local.supervision.backoff.resetPeriodMillis
Default value: 30000
mc.flow-server.source.manager.local.supervision.strategy.maxNrOfRetries
Default value: -1
mc.flow-server.source.manager.local.supervision.strategy.withinTimeRangeMillis
Default value: -1
mc.pekko-common.logging.maxLogLevelToKeep
Default value: WARNING
mc.pekko-common.logging.queueSizeThreshold
Default value: 100000
mc.pekko-common.logging.warnAfterMessagesDropCount
Default value: 500
mc.pekko-persistence-query.read-journal-class
Default value: org.apache.pekko.persistence.cassandra.query.javadsl.CassandraReadJournal
mc.pekko-persistence-query.read-journal-identifier
Default value: pekko.persistence.cassandra.query
pekko-persistence-jdbc.shared-databases.slick.db.connectionTimeout
Default value: 9000
This property controls the maximum number of milliseconds that a client (that’s you) will wait for a connection
from the pool. If this time is exceeded without a connection becoming available, a SQLException will be thrown.
1000ms is the minimum value.
Set to 9000 to ensure it’s lower than pekko.persistence.journal-plugin-fallback.circuit-breaker.call-timeout (10 seconds).
This value must be more than the validationTimeout which has a default value of 5000.
pekko.actor.allow-java-serialization
Default value: off
pekko.actor.debug.autoreceive
Default value: on
pekko.actor.debug.lifecycle
Default value: off
pekko.actor.enable-additional-serialization-bindings
Default value: on
pekko.actor.mailbox.requirements."org.apache.pekko.event.LoggerMessageQueueSemantics"
Default value: logger-mailbox
pekko.actor.provider
Default value: cluster
pekko.cluster.downing-provider-class
Default value: org.apache.pekko.cluster.sbr.SplitBrainResolverProvider
Using standard Pekko downing solution.
pekko.cluster.min-nr-of-members
Default value: 1
pekko.cluster.roles
Default value: []
pekko.cluster.seed-nodes
Default value: []
pekko.cluster.sharding.healthcheck.names
Default value: [flow]
Activate readiness check for local 'flow' shard region.
pekko.cluster.sharding.passivation.strategy
Default value: off
We do not want our sharded flows to be passivated since that will cause issues such as Kotlin script recompilation timeouts etc.
pekko.cluster.sharding.rebalance-interval
Default value: 5 s
Limiting time needed for make cluster balanced after adding new node time needed was lowered.
pekko.cluster.shutdown-after-unsuccessful-join-seed-nodes
Default value: 30s
Recommended if unable to join the discovered seed nodes - shutdown to trigger orchestrator restart (https://pekko.apache.org/docs/pekko-management/current/bootstrap/index.html#recommended-configuration
pekko.cluster.split-brain-resolver.active-strategy
Default value: keep-majority
pekko.cluster.split-brain-resolver.stable-after
Default value: 20s
pekko.cluster.use-dispatcher
Default value: cluster-dispatcher
pekko.coordinated-shutdown.exit-jvm
Default value: on
Recommended if unable to join the discovered seed nodes - shutdown to trigger orchestrator restart (https://pekko.apache.org/docs/pekko-management/current/bootstrap/index.html#recommended-configuration
pekko.discovery.config.services.local-cluster.endpoints
Default value: [{port=2552, host=127.0.0.1}]
pekko.discovery.method
Default value: config
pekko.extensions
Default value: [org.apache.pekko.cluster.metrics.ClusterMetricsExtension]
Enable metrics extension in pekko-cluster-metrics.
pekko.loggers
Default value: [org.apache.pekko.event.slf4j.Slf4jLogger]
pekko.logging-filter
Default value: org.apache.pekko.event.slf4j.Slf4jLoggingFilter
pekko.loglevel
Default value: INFO
Log level used by the configured loggers (see "loggers") as soon
as they have been started; before that, see "stdout-loglevel"
Options: OFF, ERROR, WARNING, INFO, DEBUG
pekko.management.health-checks.readiness-checks.cluster-membership
Default value: org.apache.pekko.management.cluster.scaladsl.ClusterMembershipCheck
pekko.management.http.bind-hostname
Default value: 0.0.0.0
pekko.management.http.bind-port
Default value: 19999
pekko.management.http.hostname
Default value:
pekko.management.http.port
Default value: 19999
pekko.management.http.route-providers-read-only
Default value: false
pekko.persistence.cassandra.events-by-tag.enabled
Default value: false
pekko.persistence.cassandra.journal.keyspace
Default value: akka
Name of the keyspace to be created/used by the journal. Value is "akka" for backwards compatibility.
pekko.persistence.cassandra.journal.replication-factor
Default value: 3
Replication factor to use when creating a keyspace. As we are not creating keyspace automatically by driver this
configuration will have no effect. It should be used when manually creating keyspace.
pekko.persistence.cassandra.journal.support-all-persistence-ids
Default value: off
Currently we are not executing any queries for persistence id.
In order to enable this function additional steps are required. Please read migration guide at
https://pekko.apache.org/docs/pekko-persistence-cassandra/current/#all-persistenceids-query
pekko.persistence.cassandra.journal.target-partition-size
Default value: 1000
Cassandra recommends a partition size of no more than 100M.
This setting defines how many messages Pekko stores in a partition.
The target-partition-size is closely related to the side channeling threshold (server config property: 'mc.flow-server.largeMessages.byteThreshold').
If the side channeling threshold is 100k, then this setting must be no larger than 1000, since that gives a max size of stored events of 100M.
The Pekko persistence default is set to 500.000, which in our case will result in extremely large partitions that may be unmanageable by Cassandra,
See reference.conf: https://github.com/apache/incubator-pekko-persistence-cassandra/blob/main/core/src/main/resources/reference.conf#L149.
Important notice: Changing this value requires purging of existing Cassandra partitions.
pekko.persistence.cassandra.query.max-buffer-size
Default value: 100
Used by the Pekko journal plugin to define a maximum number of events that will be fetched in one go when scanning events during recovery.
This property is closely related to the side channeling threshold (server config property: 'mc.flow-server.largeMessages.byteThreshold'),
if the default side channeling threshold is 100k, then we will fetch batches of max 10M if the max-buffer-size is set to 100.
See reference.conf: https://github.com/apache/incubator-pekko-persistence-cassandra/blob/main/core/src/main/resources/reference.conf#L229
pekko.persistence.cassandra.snapshot.keyspace
Default value: akka_snapshot
Name of the snapshot keyspace to be created/used by the journal.
Value is "akka_snapshot" for backwards compatibility.
pekko.persistence.cassandra.snapshot.read-consistency
Default value: QUORUM
pekko.persistence.cassandra.snapshot.replication-factor
Default value: 3
Replication factor to use when creating a keyspace. As we are not creating keyspace automatically by driver this
configuration will have no effect. It should be used when manually creating keyspace.
pekko.persistence.cassandra.snapshot.write-consistency
Default value: QUORUM
pekko.persistence.journal.plugin
Default value: pekko.persistence.cassandra.journal
pekko.persistence.snapshot-store.plugin
Default value: pekko.persistence.cassandra.snapshot
pekko.remote.artery.bind.hostname
Default value: 0.0.0.0
pekko.remote.artery.bind.port
Default value: 2552
pekko.remote.artery.canonical.hostname
Default value: <getHostAddress>
pekko.remote.artery.canonical.port
Default value: 2552
pekko.remote.artery.enabled
Default value: on
pekko.remote.artery.transport
Default value: tcp
pekko.remote.use-unsafe-remote-features-outside-cluster
Default value: on
Must be activated to allow watch/Terminated check to work for actor refs for dead nodes.
Needed because the flow source provisioner can get such refs from the state snapshot on recovery.
pekko.stdout-loglevel
Default value: INFO
Log level for the very basic logger activated during ActorSystem startup.
This logger prints the log messages to stdout (System.out).
Options: OFF, ERROR, WARNING, INFO, DEBUG
server.port
Default value: 8080
spring.cloud.discovery.client.composite-indicator.enabled
Default value: false
spring.jersey.servlet.load-on-startup
Default value: 1
spring.mvc.servlet.path
Default value: /mvc
2. Components
2.1. Admin
mc.flow-server.admin.environment
Default value: PLACEHOLDER
Environment artefacts are to be deployed in, must be set externally when needed
mc.flow-server.admin.fileBasedDeployment.deploymentUpdateTimeSeconds
Default value: 10
Interval in seconds that should be used while monitoring deployment folder for changes
mc.flow-server.admin.fileBasedDeployment.flowAccessCache.refreshTimeMillis
Default value: 30000
Interval in milliseconds to be used to refresh the flow access cache that stores the info about flow accesses
retrieved from the respective files in the deployment folder
mc.flow-server.admin.fileBasedDeployment.flowAccessCache.retryDelayMillis
Default value: 30000
Interval to wait before refreshing the cache when the last retrieval failed.
mc.flow-server.admin.fileBasedDeployment.rootManagementFolder
Default value: PLACEHOLDER
Absolute path to the root management folder, must be set externally when needed
2.2. Arrow
mc.flow-server.processor.flightRpcArrowWriter.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of messages to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.flightRpcArrowWriter.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.flightRpcArrowWriter.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.flightRpcArrowWriter.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.flightRpcArrowWriter.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.flightRpcArrowWriter.defaultConnectionTimeoutMillis
Default value: 5000
Currently, not in use as the Flight RPC Java implementation does not support it
mc.flow-server.processor.flightRpcArrowWriter.defaultMaxRows
Default value: 1000
mc.flow-server.processor.flightRpcArrowWriter.defaultReceiveTimeoutMillis
Default value: 20000
mc.flow-server.processor.flightRpcArrowWriter.maximumMaxRows
Default value: 10000
mc.flow-server.processor.flightRpcArrowWriter.minimumMaxRows
Default value: 1
mc.flow-server.processor.flightRpcArrowWriter.threadPool.keepAliveTimeSeconds
Default value: 60
The number of seconds before discarding idle threads (i.e. shrink the number of threads in the pool)
mc.flow-server.processor.flightRpcArrowWriter.threadPool.maximumPoolSize
Default value: 64
The maximumPoolSize
2.3. Aws
mc.flow-server.processor.awsS3ListObjects.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.awsS3ListObjects.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.awsS3ListObjects.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.awsS3ListObjects.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.awsS3ListObjects.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.awsS3ListObjects.defaultConnectionTimeoutMillis
Default value: 5000
default timeout for connection attempts from S3 client to S3 server
mc.flow-server.processor.awsS3ListObjects.defaultReceiveTimeoutMillis
Default value: 30000
default timeout to expect response from S3 server
mc.flow-server.processor.awsS3ListObjects.defaultRetryInitialDelayMillis
Default value: 5000
default initial retry timeout before next redelivery attempt
mc.flow-server.processor.awsS3ListObjects.defaultRetryMaxAttempts
Default value: 1
default max attempts S3 client will make to deliver a single message to S3 service
mc.flow-server.processor.awsS3ListObjects.defaultRetryMaxDelayMillis
Default value: 60000
default max retry timeout before next redelivery attempt. Backoff mechanism will be used to reach this value
mc.flow-server.processor.awsS3Lookup.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.awsS3Lookup.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.awsS3Lookup.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.awsS3Lookup.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.awsS3Lookup.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.awsS3Lookup.defaultConnectionTimeoutMillis
Default value: 5000
default timeout for connection attempts from S3 client to S3 server
mc.flow-server.processor.awsS3Lookup.defaultReceiveTimeoutMillis
Default value: 30000
default timeout to expect response from S3 server
mc.flow-server.processor.awsS3Lookup.defaultRetryInitialDelayMillis
Default value: 5000
default initial retry timeout before next redelivery attempt
mc.flow-server.processor.awsS3Lookup.defaultRetryMaxAttempts
Default value: 1
default max attempts S3 client will make to deliver a single message to S3 service
mc.flow-server.processor.awsS3Lookup.defaultRetryMaxDelayMillis
Default value: 60000
default max retry timeout before next redelivery attempt. Backoff mechanism will be used to reach this value
mc.flow-server.processor.awsS3ObjectMove.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.awsS3ObjectMove.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.awsS3ObjectMove.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.awsS3ObjectMove.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.awsS3ObjectMove.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.awsS3ObjectMove.defaultConnectionTimeoutMillis
Default value: 5000
default timeout for connection attempts from S3 client to S3 server
mc.flow-server.processor.awsS3ObjectMove.defaultReceiveTimeoutMillis
Default value: 30000
default timeout to expect response from S3 server
mc.flow-server.processor.awsS3ObjectMove.defaultRetryInitialDelayMillis
Default value: 5000
default initial retry timeout before next redelivery attempt
mc.flow-server.processor.awsS3ObjectMove.defaultRetryMaxAttempts
Default value: 1
default max attempts S3 client will make to deliver a single message to S3 service
mc.flow-server.processor.awsS3ObjectMove.defaultRetryMaxDelayMillis
Default value: 60000
default max retry timeout before next redelivery attempt. Backoff mechanism will be used to reach this value
mc.flow-server.processor.awsS3Writer.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.awsS3Writer.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.awsS3Writer.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.awsS3Writer.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.awsS3Writer.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.awsS3Writer.defaultConnectionTimeoutMillis
Default value: 5000
default timeout for connection attempts from S3 client to S3 server
mc.flow-server.processor.awsS3Writer.defaultReceiveTimeoutMillis
Default value: 30000
default timeout to expect response from S3 server
mc.flow-server.processor.awsS3Writer.defaultRetryInitialDelayMillis
Default value: 5000
default initial retry timeout before next redelivery attempt
mc.flow-server.processor.awsS3Writer.defaultRetryMaxAttempts
Default value: 1
default max attempts S3 client will make to deliver a single message to S3 service
mc.flow-server.processor.awsS3Writer.defaultRetryMaxDelayMillis
Default value: 60000
default max retry timeout before next redelivery attempt. Backoff mechanism will be used to reach this value
mc.flow-server.source.awsS3Consumer.defaultConnectionTimeoutMillis
Default value: 5000
default timeout for connection attempts from S3 client to S3 server
mc.flow-server.source.awsS3Consumer.defaultReceiveTimeoutMillis
Default value: 30000
default timeout to expect response from S3 server
mc.flow-server.source.awsS3Consumer.defaultRetryInitialDelayMillis
Default value: 5000
default initial retry timeout before next redelivery attempt
mc.flow-server.source.awsS3Consumer.defaultRetryMaxAttempts
Default value: 1
default max attempts S3 client will make to deliver a single message to S3 service
mc.flow-server.source.awsS3Consumer.defaultRetryMaxDelayMillis
Default value: 60000
default max retry timeout before next redelivery attempt. Backoff mechanism will be used to reach this value
mc.flow-server.source.awsS3Consumer.errorLogSkipLimit
Default value: 10
default number of errors to skip before next warning message
2.4. Azure
mc.flow-server.processor.azureBlobStorageLookup.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.azureBlobStorageLookup.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.azureBlobStorageLookup.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.azureBlobStorageLookup.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.azureBlobStorageLookup.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.azureBlobStorageLookup.defaultConnectionTimeoutMillis
Default value: 5000
mc.flow-server.processor.azureBlobStorageLookup.defaultReceiveTimeoutMillis
Default value: 20000
mc.flow-server.processor.azureBlobStorageWriter.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.azureBlobStorageWriter.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.azureBlobStorageWriter.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.azureBlobStorageWriter.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.azureBlobStorageWriter.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.azureBlobStorageWriter.defaultConnectionTimeoutMillis
Default value: 5000
mc.flow-server.processor.azureBlobStorageWriter.defaultReceiveTimeoutMillis
Default value: 20000
mc.flow-server.processor.eventHubs.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.eventHubs.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.eventHubs.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.eventHubs.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.eventHubs.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.eventHubs.retry.delayBetweenRedeliveryAttemptsInSeconds
Default value: 20
mc.flow-server.processor.eventHubs.retry.maxRedeliveryAttempts
Default value: 4
mc.flow-server.processor.eventHubs.retry.sendTimeoutInSeconds
Default value: 60
mc.flow-server.source.eventHubs.checkpointStore.azure.prefix
Default value: OVERRIDE-ME
Must be different for each deployment environment
2.5. Files
mc.flow-server.jsch.defaultJSchConfig
Default value:
mc.flow-server.processor.defaultFileWriteTimeoutMillis
Default value: 5000
mc.flow-server.processor.defaultSftpWriteMaximumReconnectAttempts
Default value: 3
mc.flow-server.processor.defaultSftpWriteSocketConnectionTimeoutMillis
Default value: 10000
mc.flow-server.processor.defaultSftpWriteTimeoutMillis
Default value: 10000
mc.flow-server.processor.defaultSmbWriteMaximumReconnectAttempts
Default value: 3
mc.flow-server.processor.defaultSmbWriteSocketConnectionTimeoutMillis
Default value: 10000
mc.flow-server.processor.defaultSmbWriteTimeoutMillis
Default value: 10000
mc.flow-server.processor.file.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.file.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.file.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.file.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.file.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.source.file.defaultFileChangedCheckIntervalMillis
Default value: 5000
mc.flow-server.source.file.defaultPollingFrequencySeconds
Default value: 30
mc.flow-server.source.file.endpointShutdownGracePeriodMillis
Default value: 10000
mc.flow-server.source.file.lineSplitModeLineMaxBytes
Default value: 100000
Maximum byte-size of a single line when the file is processed in line split mode.
100 Kilobytes
mc.flow-server.source.file.lineSplitModeMaxFileSizeBytes
Default value: 1000000000
Maximum byte-size a file can have when being processed in fragments using line split mode.
1 Gigabyte
mc.flow-server.source.file.maxFailedPollsOnWarnLevel
Default value: 20
mc.flow-server.source.file.maxFileSizeBytes
Default value: 100000000
Maximum byte-size a file can have when being processed atomically.
100 Megabyte
mc.flow-server.source.sftp.defaultFileChangedCheckIntervalMillis
Default value: 5000
mc.flow-server.source.sftp.defaultMaximumReconnectAttempts
Default value: 3
mc.flow-server.source.sftp.defaultPollingFrequencySeconds
Default value: 30
mc.flow-server.source.sftp.endpointShutdownGracePeriodMillis
Default value: 10000
mc.flow-server.source.sftp.lineSplitModeLineMaxBytes
Default value: 100000
Maximum byte-size of a single line when the file is processed in line split mode.
100 Kilobytes
mc.flow-server.source.sftp.lineSplitModeMaxFileSizeBytes
Default value: 1000000000
Maximum byte-size a file can have when being processed in fragments using line split mode.
1 Gigabyte
mc.flow-server.source.sftp.maxFailedPollsOnWarnLevel
Default value: 20
mc.flow-server.source.sftp.maxFileSizeBytes
Default value: 100000000
Maximum byte-size a file can have when being processed atomically.
100 Megabyte
mc.flow-server.source.smb.defaultFileChangedCheckIntervalMillis
Default value: 5000
mc.flow-server.source.smb.defaultMaximumReconnectAttempts
Default value: 3
mc.flow-server.source.smb.defaultPollingFrequencySeconds
Default value: 30
mc.flow-server.source.smb.endpointShutdownGracePeriodMillis
Default value: 10000
mc.flow-server.source.smb.lineSplitModeLineMaxBytes
Default value: 100000
Maximum byte-size of a single line when the file is processed in line split mode.
100 Kilobytes
mc.flow-server.source.smb.lineSplitModeMaxFileSizeBytes
Default value: 1000000000
Maximum byte-size a file can have when being processed in fragments using line split mode.
1 Gigabyte
mc.flow-server.source.smb.maxFailedPollsOnWarnLevel
Default value: 20
mc.flow-server.source.smb.maxFileSizeBytes
Default value: 100000000
Maximum byte-size a file can have when being processed atomically.
100 Megabyte
2.6. Flat File
mc.flow-server.processor.parseCsv.absoluteMaxNoOfCharactersPerColumn
Default value: 100000
Maximum number of characters allowed to be configured in ParseCsvConfig.maxNoOfCharactersPerColumn.
mc.flow-server.processor.parseHierarchicalCsv.absoluteMaxNoOfCharactersPerColumn
Default value: 100000
Maximum number of characters allowed to be configured in ParseHierarchicalCsvConfig.maxNoOfCharactersPerColumn.
2.7. General
mc.flow-server.processor.beanTransformer.expressionCacheMaxSize
Default value: 25
The bean mapper supports lookups from the messaging context using expressions.
Since the expressions can be used by custom coonverters the total set of
unique expressions can not be known up front. Therefore the expressions can not
be precompiled but are compiled lazily on first usage and cached in a FIFO cache.
This setting defines the max size this cache can have for each mapping processor.
mc.flow-server.processor.decompress.decompressionMaxSizeBytes
Default value: 100000000
The max byte-size of a decompressed archive
100MB
mc.flow-server.processor.decompress.zipArchiveMaxEntries
Default value: 100000
The max number of files in a decompressed zip archive
mc.flow-server.processor.scripting.kotlin.compilationConcurrency
Default value: 2
How many Kotlin Script compilations can be performed concurrently.
Investigations on MC-1502 concluded that this number needs to be low.
mc.flow-server.processor.scripting.kotlin.maxCompilationAttemptsOnTransientError
Default value: 3
Number of compilation attempts allowed when script fails to compile to transient error.
Example of transient error in UINT-1967
mc.flow-server.processor.scripting.kotlin.unusedCacheExpiryMillis
Default value: 60000
Cache expiration time in millis for unused scripts, minimum 1000 ms.
2.8. Google Cloud
mc.flow-server.processor.googlePubSubConsumer.defaultExecutorProviderThreads
Default value: 64
corePoolSize for the ScheduledThreadPoolExecutor
created by the executor for the google pubsub consumers.
mc.flow-server.processor.googlePubSubConsumer.defaultSystemExecutorProviderThreads
Default value: 64
corePoolSize for the ScheduledThreadPoolExecutor
created by the system-executor (used for acks and other
system actions) for the google pubsub consumers.
mc.flow-server.processor.googlePubSubProducer.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.googlePubSubProducer.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.googlePubSubProducer.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.googlePubSubProducer.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.googlePubSubProducer.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.googlePubSubProducer.defaultExecutorProviderThreads
Default value: 64
corePoolSize for the ScheduledThreadPoolExecutor
created by the executor for the google pubsub producers.
mc.flow-server.processor.googleStorageLookup.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.googleStorageLookup.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.googleStorageLookup.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.googleStorageLookup.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.googleStorageLookup.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.googleStorageLookup.defaultConnectionTimeoutMillis
Default value: 5000
Sets the timeout in milliseconds to establish a connection.
0 for an infinite timeout, a negative number will result in
the client’s default value (20000).
mc.flow-server.processor.googleStorageLookup.defaultReceiveTimeoutMillis
Default value: 30000
Sets the timeout in milliseconds to read data from an established connection.
0 for an infinite timeout, a negative number will result in
the client’s default value (20000).
mc.flow-server.processor.googleStorageWriter.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.googleStorageWriter.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.googleStorageWriter.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.googleStorageWriter.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.googleStorageWriter.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.googleStorageWriter.defaultConnectionTimeoutMillis
Default value: 5000
Sets the timeout in milliseconds to establish a connection.
0 for an infinite timeout, a negative number will result in
the client’s default value (20000).
mc.flow-server.processor.googleStorageWriter.defaultReceiveTimeoutMillis
Default value: 30000
Sets the timeout in milliseconds to read data from an established connection.
0 for an infinite timeout, a negative number will result in
the client’s default value (20000).
2.9. Habitat
mc.flow-server.source.habitat.defaultConnectionTimeoutMillis
Default value: 10000
mc.flow-server.source.habitat.defaultReconnectionDelayMillis
Default value: 10000
mc.flow-server.source.habitat.dispatcherParallelismLimit
Default value: 10
The maximum parallelism of the Kotlin Coroutines dispatcher used by a Habitat source
mc.flow-server.source.habitat.wiretap
Default value:
Activate wiretap logging for selected Habitat sources
Format: Comma separated list of <owner>--<flow-id>:<source processor id>
Example: "testowner1—testflow1:source1, testowner2—testflow2:source2"
2.10. Jdbc
mc.flow-server.processor.dbStatement.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.dbStatement.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.dbStatement.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.dbStatement.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.dbStatement.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.dbStatement.defaultExecutionTimeoutSeconds
Default value: 15
mc.flow-server.processor.dbStatement.defaultMaxRows
Default value: 1000
mc.flow-server.processor.dbStatement.maximumExecutionTimeoutSeconds
Default value: 60
mc.flow-server.processor.dbStatement.maximumMaxRows
Default value: 10000
mc.flow-server.processor.dbStatement.minimumExecutionTimeoutSeconds
Default value: 1
mc.flow-server.processor.dbStatement.minimumMaxRows
Default value: 1
2.11. Jms
mc.flow-server.processor.jms.amqp.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.jms.amqp.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.jms.amqp.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before swithcing to the half open state.
mc.flow-server.processor.jms.amqp.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.jms.amqp.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.jms.amqp.defaultConnectionCloseTimeoutMillis
Default value: 5000
mc.flow-server.processor.jms.amqp.defaultConnectionTimeoutMillis
Default value: 5000
mc.flow-server.processor.jms.amqp.defaultReceiveTimeoutMillis
Default value: 3000
mc.flow-server.processor.jms.amqp.maxReconnectAttempts
Default value: 10
Qpid will make a number of attempts to connect with provided backoff settings.
A warning will be logged every 10 connection attempts.
If used -1, Qpid will never fail and only log warnings.
We also use SingleConnectionFactory with enabled reconnection on failure.
So it is better to fail in Qpid, so that SingleConnectionFactory can start a new round of connection attempts.
mc.flow-server.processor.jms.amqp.maxRedeliveryAttempts
Default value: -1
mc.flow-server.processor.jms.amqp.reconnectBackOffMultiplier
Default value: 3
mc.flow-server.processor.jms.amqp.reconnectDelayMillis
Default value: 1000
mc.flow-server.processor.jms.amqp.useReconnectBackoff
Default value: true
mc.flow-server.source.jms.defaultReceiveTimeoutMillis
Default value: 1000
The timeout to use for receive calls
mc.flow-server.source.jms.listenerRecoverMillis
Default value: 300000
Listener container will retry after connection exception
using FastBackoff interval
2.12. Kafka
mc.flow-server.processor.kafka.batchSize
Default value: 16000
bytes
mc.flow-server.processor.kafka.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.kafka.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.kafka.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.kafka.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.kafka.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.kafka.compressionMode
Default value: ZSTD
mc.flow-server.processor.kafka.deliveryTimeoutMs
Default value: 60000
1 min
mc.flow-server.processor.kafka.linger
Default value: 500
0.5 sec
mc.flow-server.processor.kafka.maxBlockMs
Default value: 10000
10 sec
mc.flow-server.processor.kafka.maxInFlightRequests
Default value: 5
mc.flow-server.processor.kafka.reconnectBackoffMaxMs
Default value: 600000
10 min
mc.flow-server.processor.kafka.reconnectBackoffMinMs
Default value: 10000
10 sec
mc.flow-server.processor.kafka.requestTimeoutMs
Default value: 20000
20 sec
mc.flow-server.processor.kafka.requiredAcks
Default value: PARTIAL
mc.flow-server.source.kafka.authentication-error-retry-interval-ms
Default value: 60000
Interval between retries in case of authentication failure
Setting this null would disable retries
mc.flow-server.source.kafka.batch-failure-sleep-millis
Default value: 2000
We use the AcknowledgingMessageListener which enables us to process batches asynchronously.
If an error makes its way out to the endpoint we’ll nack a message. Then the whole batch
will be cancelled and a new processing attempt made after this sleep period.
Note that since the Kafka consumer is normally used with OneWay exchange pattern, errors will typically
only make their way out to the endpoint when the whole server is in a bad state.
mc.flow-server.source.kafka.executor.core-pool-size
Default value: 1
The minimum number of threads to keep in the pool, even when idling.
mc.flow-server.source.kafka.executor.max-pool-size
Default value: 256
The maximum number of threads the pool can manage at any time.
mc.flow-server.source.kafka.executor.queue-capacity
Default value: 0
Setting queue-capacity to 0 to enforce scaling the pool when needed.
Every consumer will hog a thread, so we need new consumers to spawn new threads.
Setting this to 0 mimics Executors.newCachedThreadPool() with immediate scaling of threads.
2.13. Mqtt
mc.flow-server.processor.mqtt.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.mqtt.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.mqtt.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.mqtt.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.mqtt.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.mqtt.connectionTimeoutMillis
Default value: 5000
Default value for connection timeout to MQTT broker
mc.flow-server.processor.mqtt.keepAliveSeconds
Default value: 60
Default value for keep alive interval
This will force the client to keep connection alive even if no messages appear on the broker.
To disable keep alive mechanism - use 0
mc.flow-server.processor.mqtt.threadPoolSize
Default value: 4
The number of threads for the ScheduledExecutorService set up for each processor.
Minimum value is 4.
mc.flow-server.source.mqtt.connectionTimeoutMillis
Default value: 5000
Default value for connection timeout to MQTT broker
mc.flow-server.source.mqtt.keepAliveSeconds
Default value: 60
Default value for keep alive interval
This will force the client to keep connection alive even if no messages appear on the broker.
To disable keep alive mechanism - use 0
2.14. Net
mc.flow-server.identityClient.api.url
Default value: http://metercloud-integration-identity/api
mc.flow-server.identityClient.pubKey.cache.retryDelayMills
Default value: 10000
mc.flow-server.identityClient.pubKey.cache.timeToLiveMillis
Default value: 60000
mc.flow-server.identityClient.userRole.cache.minimalRefreshThreshold
Default value: 1000
mc.flow-server.identityClient.userRole.cache.timeToLiveMillis
Default value: 60000
mc.flow-server.processor.defaultHttpConnectionTimeoutMillis
Default value: 5000
HTTP
mc.flow-server.processor.defaultHttpReceiveTimeoutMillis
Default value: 30000
mc.flow-server.processor.defaultSoapConnectionTimeoutMillis
Default value: 5000
mc.flow-server.processor.defaultSoapReceiveTimeoutMillis
Default value: 30000
mc.flow-server.processor.email.client.defaultConnectionTimeoutMillis
Default value: 5000
mc.flow-server.processor.email.client.defaultReceiveTimeoutMillis
Default value: 10000
mc.flow-server.processor.email.client.host
Default value: localhost
Might need adjustment to an actualt domain if the SMTP server complains: http://faculty.cs.niu.edu/~rickert/cf/bad-ehlo.html
mc.flow-server.processor.email.client.password
Default value: changeMe
mc.flow-server.processor.email.client.username
Default value: changeMe
mc.flow-server.processor.email.message.defaultFrom
Default value: utilihive-operataions@greenbird.com
mc.flow-server.processor.email.smtp.auth
Default value: PLAIN
mc.flow-server.processor.email.smtp.host
Default value: localhost
mc.flow-server.processor.email.smtp.port
Default value: 25
mc.flow-server.processor.email.smtp.tls
Default value: true
mc.flow-server.processor.http.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.http.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.http.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.http.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.http.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.httpClientConnectionReservationTimeoutMillis
Default value: 30000
mc.flow-server.processor.httpClientConnectionTimeToLiveSeconds
Default value: 60
mc.flow-server.processor.httpClientMaxConnectionsPerRoute
Default value: 100
mc.flow-server.processor.httpClientMaxConnectionsTotal
Default value: 10000
mc.flow-server.processor.soap.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.soap.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.soap.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.soap.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.soap.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.proxy.connectionTimeoutMillis
Default value: 20000
mc.flow-server.proxy.maxThreads
Default value: 256
mc.flow-server.proxy.rootPath
Default value: proxy/*
mc.flow-server.servlet.request.urlRegex
Default value: /()/([^/#?])(.*)
mc.flow-server.source.ownerOnTop.directAccessUrlPatterns
Default value: /rest/,/ws/,/proxy/*
mc.flow-server.source.ownerOnTop.pathsWithRestPrefix
Default value: rs,management/flows,management
mc.flow-server.source.ownerOnTop.requestURIRegex
Default value: /([^/]+)/(rs|ws|proxy|management/flows|management)(.*)
mc.flow-server.source.rest.urlRegex
Default value: /?rest/rs/()/([^/])/?(.*)
Regex that allow to split incoming REST request into FlowEntityKey and part of the path that will
be used by flow specific RestRequestProcessor to route the request.
mc.flow-server.source.soap.accessControlOnStaticResources
Default value: true
mc.flow-server.source.soap.flowIdFromRequestUrlRegex
Default value: ./ws/([^/?])
mc.flow-server.source.soap.securityFilter.urlPatterns
Default value: /ws/*
2.15. Opc
mc.flow-server.source.opc.connectionRetryFixedRateMillis
Default value: 1000
mc.flow-server.source.opc.errorLogSkipLimit
Default value: 60
mc.flow-server.source.opc.executor.keepAliveMillis
Default value: 30000
mc.flow-server.source.opc.executor.maxPoolSize
Default value: 10
2.16. Rabbitmq
mc.flow-server.processor.rabbitmq.circuitBreakerStrategy.defaultFailureThresholdMessageTotal
Default value: 10
The number of message to include when checking the failure threshold. If the number of messages that has failed
within the last failureThresholdMessageTotal messages is less than failureThresholdMinimum the breaker will
remain closed.
mc.flow-server.processor.rabbitmq.circuitBreakerStrategy.defaultFailureThresholdMinimum
Default value: 8
How many messages, out of a given total (failureThresholdMessageTotal), that has to fail in the closed state
to decide to open the breaker. If the number of failed messages is lower, the breaker will remain closed.
Must be ⇐ failureThresholdMessageTotal
mc.flow-server.processor.rabbitmq.circuitBreakerStrategy.defaultHalfOpenDelayMillis
Default value: 60000
The period to keep the breaker open before switching to the half open state.
mc.flow-server.processor.rabbitmq.circuitBreakerStrategy.defaultSuccessThresholdMessageTotal
Default value: 10
How many messages to sample in the half-open state, to compare to the minimum number of successes
(successThresholdMinimum), to decide whether to open or close the circuit breaker.
Must be >= successThresholdMinimum.
mc.flow-server.processor.rabbitmq.circuitBreakerStrategy.defaultSuccessThresholdMinimum
Default value: 8
How many messages, out of a given total (successThresholdMessageTotal), that has to be successful in the half-open
state to decide to close the breaker. If the number of successful messages is lower, the breaker will be opened again.
Must be ⇐ successThresholdMessageTotal
mc.flow-server.processor.rabbitmq.defaultConnectionTimeoutMillis
Default value: 5000
mc.flow-server.processor.rabbitmq.defaultReceiveTimeoutMillis
Default value: 5000
mc.flow-server.source.rabbitmq.backoff.initialIntervalMillis
Default value: 1000
The initial interval to wait before retrying a failed message.
mc.flow-server.source.rabbitmq.backoff.maxAttempts
Default value: 26
The maximum number of attempts to retry a failed message, 26 attempts is approximately 10 minutes.
mc.flow-server.source.rabbitmq.backoff.maxIntervalMillis
Default value: 30000
The maximum interval to wait before retrying a failed message.
mc.flow-server.source.rabbitmq.backoff.multiplier
Default value: 1.5
The multiplier to use to increase the interval after each failed retry.
mc.flow-server.source.rabbitmq.concurrentConsumers
Default value: 1
mc.flow-server.source.rabbitmq.defaultConnectionTimeoutMillis
Default value: 5000
mc.flow-server.source.rabbitmq.defaultReceiveTimeoutMillis
Default value: 5000
mc.flow-server.source.rabbitmq.listenerRecoverMillis
Default value: 30000
mc.flow-server.source.rabbitmq.prefetchCount
Default value: 10
2.17. Routing
mc.flow-server.processor.defaultDistributionStateExpiryCheckMillis
Default value: 15000
mc.flow-server.processor.defaultDistributionStateExpiryMillis
Default value: 300000
mc.flow-server.processor.defaultDistributionStateMaxMessages
Default value: 1000
mc.flow-server.processor.defaultSplitterStateExpiryCheckMillis
Default value: 15000
mc.flow-server.processor.defaultSplitterStateExpiryMillis
Default value: 300000
mc.flow-server.processor.defaultSplitterStateMaxMessages
Default value: 1000
mc.flow-server.processor.scheduledHandoff.maxDelayMillis
Default value: 7776000000
Maximum period a message could be scheduled for delivery
90 days
mc.flow-server.processor.scheduledHandoff.maxNoOfScheduledMessages
Default value: 100000
Max number of messages that could be scheduled simultaneously
mc.flow-server.processor.scheduledHandoff.pollingIntervalMillis
Default value: 1000
Frequency of checking if there are persisted messages to be forwarded at the moment
mc.flow-server.processor.scheduledHandoff.shutdownTimeoutSeconds
Default value: 10
Timout waiting for shutdown to complete
mc.flow-server.processor.scheduledHandoff.supervision.backoff.maxBackOffMillis
Default value: 60000
the exponential back-off is capped to this duration
mc.flow-server.processor.scheduledHandoff.supervision.backoff.minBackOffMillis
Default value: 1000
minimum duration until scheduledHandoff actor will be started again, if it is terminated
mc.flow-server.processor.scheduledHandoff.supervision.backoff.resetPeriodMillis
Default value: 30000
the back-off is reset if the child does not crash within this duration
mc.flow-server.processor.scheduledHandoff.supervision.maxNrOfRetries
Default value: -1
the number of times scheduledHandoff actor can be restarted, negative value means no limit,
if the limit is exceeded the child actor is stopped
mc.flow-server.processor.windowedAggregationHandoff.compactionIntervalMinutes
Default value: 5
Frequency of checking if there are aggregated messages that can be deleted for compaction
mc.flow-server.processor.windowedAggregationHandoff.retryIntervalMinutes
Default value: 5
Frequency of checking if there are aggregated messages that need to be retried
mc.flow-server.processor.windowedAggregationHandoff.shutdownTimeoutSeconds
Default value: 10
Timout waiting for shutdown to complete
mc.flow-server.processor.windowedAggregationHandoff.supervision.backoff.maxBackOffMillis
Default value: 60000
the exponential back-off is capped to this duration
mc.flow-server.processor.windowedAggregationHandoff.supervision.backoff.minBackOffMillis
Default value: 1000
minimum duration until windowedAggregationHandoff actor will be started again, if it is terminated
mc.flow-server.processor.windowedAggregationHandoff.supervision.backoff.resetPeriodMillis
Default value: 30000
the back-off is reset if the child does not crash within this duration
mc.flow-server.processor.windowedAggregationHandoff.supervision.maxNrOfRetries
Default value: -1
the number of times windowedAggregationHandoff actor can be restarted, negative value means no limit,
if the limit is exceeded the child actor is stopped