Multi-Flow Design

Multi-flow design combines multiple flows to handle scenarios that a single flow cannot handle effectively, such as fan-out to multiple destinations, batch splitting, and content-based routing.

This page introduces key patterns and the processors that implement them. For a full reference of processor properties, follow the links to the relevant processor documentation.

Distribution

Scenario

Consider a Sales system that receives an order event and must forward it to Finance, Delivery, and Reporting systems. A single-flow implementation places all three restRequest processors sequentially in one pipeline:

flowConfig {
    id = "order-flow"
    ownerId = OWNER_ID
    exchangePattern = RequestResponse
    restApi { id = "order-api"; ... }
    restRequest { id = "finance";   defaultMethod = POST; address = URL("https://OVERRIDE_ME/finance") }
    restRequest { id = "delivery";  defaultMethod = POST; address = URL("https://OVERRIDE_ME/delivery") }
    restRequest { id = "reporting"; defaultMethod = POST; address = URL("https://OVERRIDE_ME/reporting") }
}

This design tightly couples the downstream systems and can unintentionally pass downstream responses between processors.

The distribute Processor

The distribute processor can forward the same message to multiple target flows simultaneously. Distribution is not complete until all target flows have confirmed reception. The distributor keeps non-persistent state to avoid re-delivering to targets that already confirmed, so any delivery guarantee must be implemented upstream in the main flow.

Main Flow
  restApi (source)
  distribute -> Target Flow A
             -> Target Flow B
             -> Target Flow C

Target Flow Types

Each flow referenced by distribute can be any flow type. The two common choices are:

OneWay + handoff flow

Use this when targets should be decoupled from the main flow and from each other. The main flow completes once each message is handed off to the target source. Each target has its own persistence, redelivery, and dead letter handling.

Headless flow

Use this when the main flow must wait for all target processing to complete. The target pipeline is inlined in the main flow and shares the main flow exchange pattern and reliability configuration.

OneWay + handoff Headless

Main flow waits for target to…​

Accept the message at the source

Complete full target pipeline

Own delivery guarantee?

Yes - own redelivery + dead letter

No - owned by main flow

Own error handling scope?

Yes

No

Persistence

Message is persisted again at the handoff source — a second write to reliable storage per distributed message

No additional persistence — processed within the main flow’s persistence scope

Typical use

Decoupled independent downstream systems

Synchronous fan-out where client needs all results
Each OneWay + handoff target flow persists every incoming message before processing — this is what gives it an independent delivery guarantee. When distributing to multiple handoff target flows, each target flow persists the message independently and in parallel. This is an inherent part of the decoupling model: if independent persistence per target is not needed, Headless target flows avoid this overhead at the cost of sharing the main flow’s delivery guarantee.

Worked Example - Fan-Out to Independent Systems

// Main flow: receives and distributes
val orderDistributionSpec = flowConfig {
    id = "order-distribution-api"
    ownerId = OWNER_ID
    exchangePattern = RequestResponse
    restApi {
        id = "order-api"
        apiSpecId = orderRestResourceKey.toResourceIdentifier()
    }
    distribute {
        id = "distribute-order"
        flowId(FLOW_ID_FINANCE)
        flowId(FLOW_ID_DELIVERY)
        flowId(FLOW_ID_REPORTING)
    }
    map {
        id = "create-response"
        mapSpec = """{ "message": "Distributed" }""".trimIndent()
    }
}

// Finance target flow: processes finance requests
val financeFlowSpec = flowConfig {
    id = FLOW_ID_FINANCE
    ownerId = OWNER_ID
    exchangePattern = OneWay
    handoff { id = "handoff-source" }
    restRequest {
        id = "finance-request"
        defaultMethod = POST
        address = URL("https://OVERRIDE_ME/finance")
    }
}

// Delivery and Reporting target flows: same pattern

Worked Example - Fan-In from Multiple Sources

The inverse pattern is also supported: multiple source flows distribute to a shared handoff flow. Consider a scenario where you need to consolidate data from several weather services. Some of these services provide XML, while others provide CSV files. In both cases, the data must be converted to JSON before reaching the final destination, which is a database.

The source flows can use different connectors to retrieve data and perform the necessary format conversions before distributing to the shared handoff flow. The shared handoff flow then performs the final transformation and writes to the database target. A dead letter strategy on the shared handoff flow can capture database write failures.

Key Design Implications

  • Decoupling between main and target flows and among targets: With OneWay + handoff, slow or failed target processing does not block the main flow or other targets.

  • Non-persistence and idempotency: Because distributor state is non-persistent, redelivery can produce duplicates.

  • Main flow result: distribute returns one representative response (not an aggregate), in this order:

    1. Non-transient failure (any target)

    2. Transient failure (any target)

    3. Successful response

    4. Filtered message

Using RequestResponse on a handoff flow is a rare pattern that increases latency and resilience complexity. See handoff for full implications.

for more information, see distribute and handoff.

Splitting Collections

Scenario

The Sales system sends a batch (for example ['order1', 'order2']), but Finance only accepts one order per request.

The split Processor

split divides a collection payload into individual fragments and sends each fragment as a separate message to one target flow. Processing completes only when every fragment has a response. Like distribute, splitter state is non-persistent.

Main Flow
  restApi (source)
  setPayload (collection)
  split -> Target flow (one fragment per invocation)
By default, each element in the collection becomes one fragment — a collection of 10 orders produces 10 fragments. Set chunkSize to N to group elements into list-fragments instead: each fragment then contains a list of at most N elements, and the last fragment may contain fewer. For example, chunkSize = 3 on a collection of 10 orders produces 4 fragments: three containing 3 orders each and one containing the remaining 1. Use this when the target system accepts small batches but not individual records.

Target Flow Exchange Pattern

OneWay + handoff flow

The main flow receives handoff receipts immediately after each fragment is handed off to the target source; the target flow processes fragments independently.

val financeTargetSpec = flowConfig {
    id = FLOW_ID_FINANCE_TARGET
    ownerId = OWNER_ID
    exchangePattern = OneWay
    handoff { id = "order-source" }
    restRequest {
        id = "send-to-finance"
        defaultMethod = POST
        address = URL("https://OVERRIDE_ME/finance")
    }
}

Headless flow

The main flow waits for all fragments to finish and for all responses to return. With aggregate at the end of the target pipeline, split returns a composite result.

val financeTargetSpec = flowConfig {
    id = FLOW_ID_FINANCE_TARGET
    ownerId = OWNER_ID
    exchangePattern = Headless
    restRequest {
        id = "send-to-finance"
        defaultMethod = POST
        address = URL("https://OVERRIDE_ME/finance")
    }
    aggregate {
        id = "collect-results"
        maxNoOfMessagesToKeep = 100
        maxAgeInSecondsForMessagesToKeep = 600
    }
}
// Result shape:
// { "numberOfFragments": 2, "fragments": ["result1", "result2"] }

The Role of aggregate - Only When Needed

Use aggregate only in these cases:

Case 1 - Main flow needs collected fragment results

Use a Headless target + aggregate to return { "numberOfFragments": N, "fragments": […​] }.

Case 2 - Post-split processing must run after all fragments finish

Use a OneWay target + handoff and place aggregate as a synchronization point.

val financeTargetSpec = flowConfig {
    id = FLOW_ID_FINANCE_TARGET
    ownerId = OWNER_ID
    exchangePattern = OneWay
    handoff { id = "order-source" }
    restRequest {
        id = "send-to-finance"
        defaultMethod = POST
        address = URL("https://OVERRIDE_ME/finance")
    }
    aggregate {
        id = "sync-point"
        maxNoOfMessagesToKeep = 100
        maxAgeInSecondsForMessagesToKeep = 600
    }
    restRequest {
        id = "send-report"
        defaultMethod = POST
        address = URL("https://OVERRIDE_ME/reporting")
    }
}
Cleaner alternative when the aggregated result is not needed: use a Headless target without aggregate and place post-split logic directly in the main flow after split. 
flowConfig {
    id = "main-flow"
    ownerId = OWNER_ID
    exchangePattern = OneWay
    restApi { id = "order-api"; ... }
    setPayload { id = "create-list"; source = "envelope.payload.value" }
    split { id = "split-orders"; flowId = FLOW_ID_FINANCE_TARGET }
    restRequest { id = "send-report"; ... }
}

val financeTargetSpec = flowConfig {
    id = FLOW_ID_FINANCE_TARGET
    ownerId = OWNER_ID
    exchangePattern = Headless
    restRequest { id = "send-to-finance"; ... }
}

Key Design Implications

  • Decoupling between main and target flows: With OneWay + handoff, slow or failed target processing does not block the main flow.

  • Non-persistence and idempotency: Splitter state is non-persistent, so duplicates can occur on redelivery.

  • Main flow result: Similar to distribute, split returns one representative response. The { "numberOfFragments": N, "fragments": […​] } composite result is only returned when aggregate is used in a Headless target flow.

For more information, see split and aggregate.

Content-Based Routing

Overview

Content-based routing directs messages to different target flows based on their content. In Connect, this is implemented using the distribute processor with targetFilterExpression.

Filter expressions run in the Standard Expression Context extended with one additional property: targetFlowId — the ID of the flow currently being evaluated. This means expressions can filter on the payload directly, on exchange properties, on the target flow ID, or on any combination of these.

Example - Routing by Payload Content (Shipping vs Finance)

The simplest approach — reference the payload field directly:

distribute {
    id = "distribute-message"
    flowId(FLOW_ID_SHIPPING)
    flowId(FLOW_ID_FINANCE)
    targetFilterExpression(
        "envelope.payload.value == 'address' and targetFlowId.contains('shipping')"
    )
    targetFilterExpression(
        "envelope.payload.value == 'price' and targetFlowId.contains('finance')"
    )
}

Alternatively, extract the routing key first with setExchangeProperty. This approach is useful when the same key is referenced in multiple expressions or the payload path is deeply nested:

setExchangeProperty {
    id = "set-exchange-property"
    propertyName = "filter"
    source = "envelope.payload.value"
}
distribute {
    id = "distribute-message"
    flowId(FLOW_ID_SHIPPING)
    flowId(FLOW_ID_FINANCE)
    targetFilterExpression(
        "exchangePropertiesMap['filter'] == 'address' and targetFlowId.contains('shipping')"
    )
    targetFilterExpression(
        "exchangePropertiesMap['filter'] == 'price' and targetFlowId.contains('finance')"
    )
}

Unmatched Messages

If all targetFilterExpression entries evaluate to false, the message is not forwarded to any target flow. The message is filtered out, and no error is raised.

To route unmatched messages to all target flows:

targetFilterExpression("envelope.payload.value != 'address' and envelope.payload.value != 'price'")

To route all messages to one specific target regardless of content:

targetFilterExpression("targetFlowId.contains('my-target')")

The filter Processor

The filter processor cancels a message if its expression evaluates to false, stopping all further processing in the flow. Use it before distribute when messages that do not match any routing criteria should be rejected entirely — before distribution is attempted.

filter {
    id = "filter-message"
    expression = "envelope.payload.value == 'address' or envelope.payload.value == 'price'"
}

What happens on a false result depends on the flow’s exchange pattern:

  • In a RequestResponse flow, the client receives a cancellation response with a reason field:

    { "reason": "Message did not match filter expression: envelope.payload.value == 'address' or envelope.payload.value == 'price'" }

  • In a OneWay flow, the message is silently canceled. No error is raised, and no dead letter is produced.

filter vs targetFilterExpression

filter and targetFilterExpression serve different purposes and can be used together:

filter targetFilterExpression

Scope

Cancels the message for the entire flow

Filters per target flow ID inside distribute

Use when

The message should not be processed at all if it does not match

The message should reach some targets but not others

For more information, see distribute, filter, setExchangeProperty and Standard Expression Context.

Choosing the Right Pattern

Scenario Processor Target flow type Notes

One message → multiple destinations, decoupled processing

distribute

OneWay + handoff

Each target operates independently; failure in one does not affect others.

One message → multiple destinations, main flow waits for all

distribute

Headless

Main flow blocked until all targets complete; result is representative.

Multiple source flows → single shared destination

distribute (from each source)

OneWay + handoff

Fan-in pattern; shared handoff flow consolidates from multiple sources.

Collection → process each item, decoupled from main flow

split

OneWay + handoff

No aggregate needed.

Collection → process each item, main flow needs all results

split

Headless + aggregate

aggregate collects all fragment responses.

Collection → process each item, then run post-split logic

split

OneWay + handoff + aggregate (sync point)

aggregate used for synchronization only; result is not consumed.

Collection → process each item, then run post-split logic

split

Headless, no aggregate

Place further processing in main flow after split; cleaner than sync point pattern.

Route by message content to different destinations

distribute + targetFilterExpression

OneWay or Headless

Reference payload fields directly, or use setExchangeProperty to extract a routing key when you reuse the same key or the payload path is deeply nested.