Enterprise Integration Patterns
Messaging Patterns
HOME PATTERNS RAMBLINGS ARTICLES TALKS DOWNLOAD BOOKS CONTACT
Messaging Patterns
Scatter-GatherScatter-GatherMessaging Patterns » Message Routing

In the order processing example introduced in the previous patterns , each order item that is not currently in stock could be supplied by one of multiple external suppliers. However, the suppliers may or may not have the respective item in stock themselves, they may charge a different price and may be able to supply the part by a different date. To fill the order in the best way possible, we should request quotes from all suppliers and decide which one provides us with the best term for the requested item.

How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?

Use a Scatter-Gather that broadcasts a message to multiple recipients and re-aggregates the responses back into a single message

The Scatter-Gather routes a request message to the a number of recipients. It then uses an Aggregator to collect the responses and distill them into a single response message.

There are two variants of the Scatter-Gather that use different mechanisms to send the request messages to the intended recipients:

... Read the entire pattern in the book Enterprise Integration Patterns

Example: Serverless Loan Broker on AWSNEW

The serverless Loan Broker example includes a Scatter-Gather that routes requests for a loan quote to interested banks and selects the best offer from the incoming responses. The example implements both a a Recipient List using AWS Step Functions' Map state (see Part 2) and a Publish-Subscribe Channel using the Amazon Simple Queuing Service (SQS) (see Part 3).

Example: Mulesoft ESBNEW

Mulesoft's enterprise service bus includes a Scatter-Gather construct, described as Scatter-Gather Router and can be configured using the scatter-gather element.

Although each route is independently executed, the Scatter-Gather Router configures a pre-defined number of paths, so it doesn't broadcast to an unknown number of recipients. Knowing all recipients makes the completeness condition of "Wait for all" (see Aggregator) a natural choice. Route errors propagate to the Scatter-Gather, which in turn flags an error, passing on any results that were successfully passed by the individual routes. The aggregation strategy is concatenation, turning single variable values into a list of all route results.

Example: Azure Durable Functions (Fan out/fan in)NEW

Durable functions, which are used to orchestrate serverless Azure Functions, make it easy to implement Scatter-Gather, following the Fan out/fan in construct. which is supported by the WhenAll method, which waits for all functions to execute:

Fan out/fan in with Azure Durable Functions

def orchestrator_function(context: df.DurableOrchestrationContext):
    work_batch = yield context.call_activity("GetVendors", None)
    parallel_tasks = [ context.call_activity("Vendor", b) for b in work_batch ]
    outputs = yield context.task_all(parallel_tasks)
    total = sum(outputs)
  

The number of recipients can be calculated at run-time, and the completeness condition can support a variety of strategies, including "Wait for all" (Task.WhenAll), "First Best" (Task.WhenAny), or "Fixed Timeout" (Task.WaitAll).

Related patterns:

Aggregator, Introduction to Composed Messaging Examples, Asynchronous Implementation with MSMQ, Asynchronous Implementation with TIBCO ActiveEnterprise, Composed Message Processor, Publish-Subscribe Channel, Recipient List, Return Address, Splitter


Creative Commons Attribution License

You can reuse the following elements under the Creative Commons Attribution license: pattern icon, pattern name, problem and solution statements (in bold), and the sketch. Other portions are protected by copyright.

Enterprise Integration Patterns book cover

Enterprise Integration Patterns
The classic, as relevant as ever. Over 90,000 copies sold.

Software Architect Elevator book cover

The Software Architect Elevator
Learn how architects can play a critical role in IT transformation by applying their technical, communication, and organizational skills.

Cloud Strategy book cover

Cloud Strategy
Fill the large gap between high-level goals and product details by understanding decision trade-offs.