Enterprise Integration Patterns Enterprise Integration Patterns
HOME    PATTERNS    RAMBLINGS    ARTICLES    TALKS    DOWNLOAD    BOOKS    CONTACT

My ongoing thoughts about the present and future of integration, SOA and Web services. [see all]
37 Things or "Where have all my ramblings gone?"
25 Years of OOP
How to Scale an Organization? The same way you scale a system!
YOW!
Dec 1-9, 2016
Sydney, Melbourne, Brisbane
OOP
Jan 30- Feb 3, 2017
Munich
A Decade of Enterprise Integration Patterns
(IEEE Software)
20 Years of Patterns' Impact
(IEEE Software)
Conversations Between Loosely Coupled Services
(Video on InfoQ)
Developing in a Service-oriented World
(Video on InfoQ)
SOA Patterns - New Insights or Recycled Knowledge?
(Whitepaper)
Let's Have a Conversation
(IEEE Internet Computing)
Programming Without a Call Stack - Event-driven Architectures
(ObjektSpektrum, 2006)
Your Coffee Shop Does Not Use Two-Phase Commit
(IEEE Software)
Developing in a Service-Oriented World
(ThoughtWorks Whitepaper)
An Asynchronous World
(Software Development)
Nearfield Communication (NFC) in Japan
(OOP 2012)
Embedded OSS
(OOP 2012)
Distributed Programming the Google Way
(Devoxx 2009)
Developing in a Service-Oriented World
(SOACon 2007)
Programming without a Call Stack: Event-driven Architectures
(SOACon 2007)
Software Visualization and Model Extraction
(TheServerSide)
Conversations Between Loosely Coupled Systems
(SD West, 2006)
Enterprise Integration Patterns
(JAOO, 2003)
Microsoft MVP
 
Hillside
 
IEEE Software Board Member

Patterns and Best Practices for Enterprise Integration

Buy on

Today's applications rarely live in isolation. Users expect instant access to all functions, which may be provided by disparate applications and services, inside or outside the enterprise. Integrating applications and services remains more difficult than it should be, though: developers have to deal with asynchrony, partial failures, and incompatible data models. The lack of a common vocabulary and body of knowledge for asynchronous messaging architectures made it difficult to avoid common pitfalls.

That's why Bobby Woolf and I documented a pattern language consisting of 65 integration patterns to establish a technology-independent vocabulary and a visual notation to design and document integration solutions, ranging from connecting applications to a messaging system, routing messages to the proper destination, and monitoring the health of a messaging system. Each pattern not only presents a proven solution to a recurring problem, but also documents common "gotchas" and design considerations. The patterns are brought to life with examples implemented in messaging technologies, such as JMS, SOAP, MSMQ, .NET, and other EAI Tools. The solutions are relevant for a wide range of integration tools and platforms, such as IBM WebSphere MQ, TIBCO, Vitria, WebMethods (Software AG), or Microsoft BizTalk, messaging systems, such as JMS, WCF, Rabbit MQ, or MSMQ, ESB's such as Apache Camel, Mule, WSO2, Oracle Service Bus, Open ESB, SonicMQ, Fiorano or Fuse ServiceMix.

Buy the book Enterprise Integration Patterns or read a sample chapter first. All content on this site is original and is maintained by Gregor Hohpe. Find the most recent content in my blog or articles. Please contact me if you have feedback or would like me to speak at your company or event.

"The core language of EAI, defined by Gregor Hohpe and Bobby Woolf, is also the core language of defining ESB flows and orchestrations, as seen in the ESB's developer tooling."
--Forrester Research [More Testimonials]

New Book: Enterprise Transformation

Buy on

As enterprise integration remains a relevant topic some 13 years after Enterprise Integration Patterns was published, digital transformation has become a major consideration for large enterprises: they are under pressure from so-called "digital disruptors" who attack unexpectedly with brand-new business models, operate free of legacy, and release enhanced products on a weekly basis. To compete in the digital age, traditional enterprises must transform not only their business model but also their IT departments.

I collected my experience as chief architect into a book to help IT architects or CTOs combine superb technical, communication, and organizational skill to successfully drive IT transformation in large organizations. The book titled 37 Things One Architect Knows About IT Transformation is available as DRM-free E-Book on leanpub.com or as print book on Amazon.

Work-in-progress: Conversation Patterns

Asynchronous messaging is the foundation for most integration solution because its architectural style acknowledges the challenges of distributed communication, such as latency or partial failure. However, many interactions between systems extend beyond sending a single, stateless message: a request may expect a response; a handshake or authentication are needed first; a reservation is confirmed or expires. Such conversations, stateful exchanges between participants, present new design challenges and patterns. I therefore started documenting Conversation Patterns, which are the starting point for Enterprise Integration Patterns 2.

What Makes Integration Difficult?

Architecting integration solutions is a complex task. There are many conflicting drivers and even more possible 'right' solutions. Whether the architecture was in fact a good choice usually is not known until many months or even years later, when inevitable changes and additions put the original architecture to test. Unfortunately, there is no "cookbook" for enterprise integration solutions. Most integration vendors provide methodologies and best practices, but these instructions tend to be very much geared towards the vendor-provided tool set and often lack treatment of the bigger picture, including underlying guidelines, principles and best practices.

Asynchronous Messaging Architectures

Asynchronous messaging architectures have proven to be the best strategy for enterprise integration because they allow for a loosely coupled solution that overcomes the limitations of remote communication, such as latency and unreliability. That's why most EAI suites and ESB's are based on asynchronous messaging. Unfortunately, asynchronous messaging is not without pitfalls. Many of the assumptions that hold true when developing single, synchronous applications are no longer valid. Vendor-independent design guidance helps developers avoid these pitfalls so they can build robust integration architectures based on asynchronous messaging.

How can Patterns Help?

Patterns are a proven way to capture experts' knowledge where no simple “one size fits all” answers exist, for example in application architecture, object-oriented design, or message-oriented integration . Each pattern tackles a specific problem by discussing design considerations and presenting an elegant solution that balances often conflicting forces. The solution is not the first approach that comes to mind, but one that has evolved through actual use over time, capturing the experience that senior developers and architects have gained by repeatedly building solutions and learning from their mistakes.

What am I Reading Right Now?

Designing Delivery: Rethinking IT in the Digital Service Economy, Sussna, O'Reilly 2015

Jeff strikes a great balance between drawing on theoretical foundations (cybernetics, complex systems theory) and real-world examples to explain why and how traditional IT organizations must transform to support the business in a digital world. A book to hand to all IT managers.

DevOps: A Software Architect's Perspective, Bass, Weber, Zhu, Addison-Wesley 2015

This book takes the (cloud) architecture viewpoint on DevOps. SEI titles can be a bit encyclopedic, but are thorough and this one is refreshingly close to real-world cloud solutions and tooling. A must read if you deal with development organizations that are moving (or should be moving) to the cloud.

The DevOps Handbook, Gene Kim et al, IT Revolution Press 2016

Hot off the press, the IT Revolution crowd shared their wisdom and experience implementing DevOps from understanding the fundamental mechanisms to overcoming inevitable obstacles.

Table of Contents
Preface
Introduction
Solving Integration Problems using Patterns
Integration Styles
File Transfer
Shared Database
Remote Procedure Invocation
Messaging
Messaging Systems
Message Channel
Message
Pipes and Filters
Message Router
Message Translator
Message Endpoint
Messaging Channels
Point-to-Point Channel
Publish-Subscribe Channel
Datatype Channel
Invalid Message Channel
Dead Letter Channel
Guaranteed Delivery
Channel Adapter
Messaging Bridge
Message Bus
Message Construction
Command Message
Document Message
Event Message
Request-Reply
Return Address
Correlation Identifier
Message Sequence
Message Expiration
Format Indicator
Interlude: Simple Messaging
JMS Request/Reply Example
.NET Request/Reply Example
JMS Publish/Subscribe Example
Message Routing
Content-Based Router
Message Filter
Dynamic Router
Recipient List
Splitter
Aggregator
Resequencer
Composed Msg. Processor
Scatter-Gather
Routing Slip
Process Manager
Message Broker
Message Transformation
Envelope Wrapper
Content Enricher
Content Filter
Claim Check
Normalizer
Canonical Data Model
Interlude: Composed Messaging
Synchronous (Web Services)
Asynchronous (MSMQ)
Asynchronous (TIBCO)
Messaging Endpoints
Messaging Gateway
Messaging Mapper
Transactional Client
Polling Consumer
Event-Driven Consumer
Competing Consumers
Message Dispatcher
Selective Consumer
Durable Subscriber
Idempotent Receiver
Service Activator
System Management
Control Bus
Detour
Wire Tap
Message History
Message Store
Smart Proxy
Test Message
Channel Purger
Interlude: Systems Management Example
Instrumenting Loan Broker
Integration Patterns in Practice
Case Study: Bond Trading System
Concluding Remarks
Emerging Standards
Appendices
Bibliography
Revision History