In general, writing software is made hard by our inability to predict the future. We’re always caught between the two stools of building what we need now and what we might need in the future. Writing infrastructure is even harder because it’s an expression of common patterns and challenges of implementation in a specific context such as one company’s systems or services. The broader the context (e.g. all enterprises), the harder it gets to account for all possible permutations of usage.

So to a core trouble with building infrastructure: common patterns and challenges of implementation can only be established by reviewing history. Basically until some systems have been built, we can’t tell with certainty what the infrastructure should look like.

It’s all too easy to believe that some problem we’re currently faced with is generic and therefore best tackled by:

How do we know something is generic? Experience. To be confident something is generic we must have seen it across our systems universe (that is the collection of systems in our domain of concern). The danger is that multiple teams adopt their own solution to the same problem but this isn’t necessarily a bad thing. Each team is doing valuable investigation work that will help identify the most promising options for a solution. The trick of course is to have sufficient cross-team discussion about architecture so as to avoid excessive proliferation of independent solutions.

I have a little mantra that I use to remind myself and others of this thorny issue: Experience leads infrastructure.

Technorati Tags: architecture, philosophy, systems