Gregor's Ramblings

Configure This!

July 5, 2005

Gregor Hohpe
Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I like to work on and write about asynchronous messaging systems, service-oriented architectures, and all sorts of enterprise computing and architecture topics. I am also a technical director in Google Cloud's Office of the CTO.

Having been to a number of conferences and their attached expo floors recently it was not surprising that I was not able to escape the line "this tool does not require programming, everything is done through configuration". Hearing this tag line for the third or fourth time got me wondering, "what really is the difference between coding and configuring?"

Good Intentions

Just to be sure, this is not intended to be vendor bashing (even though it can sure be tempting sometimes :-) ). Seriously, buying tools can save a lot of development work that we can buy off the shelves. These tools make our lives easier because they do a lot of the grunt work for us. Libraries and tools also serve an important community function by widening the reach of a specific technology. Yeah, a lot of us guys who like to have dinner with the Microsoft Indigo team or the Java OSS crowd for fun sure don't mind dabbling with WSDL elements and writing our own XML serializers. However, if your main task is to solve business problems and your boss won't let you go to TechEd because he (or she) considers it a waste of time, having a tool that lets you stay out of the nitty gritty can be a lifesaver.

Levels of Abstraction

The primary technique of making developers' lives easier is to raise the level of abstraction at which they have to work. Very few of us still write assembly code, read individual file blocks from a disk, or put individual packets onto the network. This level of detail has been nicely wrapped behind high-level languages, file and socket streams. These programming abstractions are very convenient (try doing without them!) and dramatically increase productivity. All I have to do is know a file name or a URI and bytes come streaming in and out. Of course, if abstractions are this useful, the question is whether we can add new abstraction layers to increase productivity further. If we are able to provide enough abstraction, we might be able to do away with coding and can create a solution simply by configuration. Or can we?

Variability Points

In our quest to provide more and more abstraction and to simplify the programming model further and further we come up to an interesting challenge: We want to simplify our world but we do not want to lose too much flexibility. If in fact I want to store individual data records on my hard disk (for example to create a rapid direct-access indexing system) the file stream abstraction does not do me much good. In order to make our life really simpler it is not enough to just paint a thin visual veneer over the existing programming interface. Instead we have to actively reduce complexity and therefore flexibility. This exercise is quite similar to the architecture guessing game I talked about before. If we take away the wrong things, our abstraction is limited in its usefulness. If we take away too few things we did not accomplish much in terms of simplifying our (programming) life.

Domain-Specific Languages

There are two ways out of the abstraction and variability conundrum. You can try to get more people to think along your lines so that they are less likely to be wanting the features that you eliminated when you created the abstraction. This is the approach of the evangelists and, to some extend, the patterns authors. Alternatively, you can avoid limiting yourself to a single general-purpose abstraction. Instead you could provide different abstractions for different purposes. This approach is great as long as you can find a way to make creating these abstractions easy, which is usually not the case. This is the very problem the Domain-Specific Language (DSL) folks are trying to solve by providing Language Workbenches. The workbenches are tools to create new languages that provide specific abstractions. DSLs are still in very early stages but promise to become quite useful.

Configuration or Programming?

But back to our original question. What kind of abstraction deserves to be called "configuration" versus "high-level programming"? We are often led to believe that everything that sits in an XML file can be considered configuration. This can be true for very simple scenarios, but XSLT programs are also XML files, and those things are definitely programming, and not for the faint of heart at that. A better criterion that comes to mind is whether what you create is executable. If the algorithm is predefined and I only supply a few key values this is more likely to be considered configuration. On the other hand, calling a method with parameters is not really any different and is certainly considered programming. Also, a "configuration" value can decide between different algorithms, so it has some impact on the program execution. Many development platforms actually allow us to accomplish the same tasks via configuration or code, depending on our preference (Microsoft Indigo is a good example where this is done fairly consistently). So how about whether what you enter is executed in a specific order, i.e. is what you enter purely data or is it more like a set of sequential instructions? This looks like a reasonable distinction, even though there are some definite gray areas, especially as we increasingly deal with declarative programming languages, such as languages to configure rules engines or our favorite language XSLT. Declarative languages focus on specifying relationships between constructs and do typically not presuppose a specific execution order like procedural languages. Also, aren't coding instructions just data for the execution engine? Apparently, it is mostly a matter of perspective.

When you make a system configurable I think it is useful to ask yourself whether you are really just configuring or whether you are programming at a high level of abstraction. Considering your mechanism as programming can remind you that you should allow for proper tool support like editors or validation. One of the best examples I have seen is in messaging architectures. We created a set of individual components that can be composed into a larger solution by connecting them with channels. "Configuration" files would determine which channel(s) a component receives messages from and which channel(s) it sends message to. This kind of setup is quite common in EAI tools where adapters and such are configured to talk to each other via specific channel names. In our case, the channel names were sitting in some local XML configuration files because it was convenient to read data from there and we did not really think much about it. After a while it turned out, though, that a lot of errors in the system were due to misconfiguration. Components would not talk to each other because the channel names were misconfigured or the flow of messages was wrong because we chained together the right components in the wrong order. This started to get us thinking that what we created was not a simple configuration file but really a high-level programming model for the composition layer of our system. Ending the shadow life of the config files and making them first class citizens caused us to check these files into source control like any other source file, and to create a set of validation and management tools to perform efficient debugging and troubleshooting (see my related earlier post) . It also reminded us that providing this "configuration" is useful but it does not automatically mean that it is trivial or does not require skilled developers.

Configuration Progamming

Now that I am back on-line I can include a link to a dictionary entry on Configuration Programming. Here it states that:

Configuration Programming: An approach that advocates the use of a separate configuration language to specify the coarse-grain structure of programs. Configuration programming is particularly attractive for concurrent, parallel and distributed systems that have inherently complex program structures.

This definition highlights that academia have been thinking about the gray area between configuration and programming for some time. One example of a language designed specifically for the described purpose is Darwin.


Abstractions are a good thing, but let's not get carried away thinking that calling the abstraction "configuration" is suddenly going to eliminate the need for developers. Instead, treat configuration as a first-class citizen that requires design, testing, version control, and deployment management.


Follow:       Subscribe  SUBSCRIBE TO FEED


Gregor is a technical director in Google Cloud's Office of the CTO. He is a frequent speaker on asynchronous messaging and service-oriented architectures and co-authored Enterprise Integration Patterns (Addison-Wesley). His mission is to make integration and distributed system development easier by harvesting common patterns and best practices across many different technologies.