Pragmatic Dictator

Check out this job spec.

Notice anything interesting? It’s for a seriously heavyweight distributed systems engineer sure but look deeper. Do you see mention of a single piece of technology? J2EE? JavaScript? Ruby? No, right? How weird is that? How many job specs do you see like that? Surely what matters is whether you know JBoss or Websphere, Java or Erlang or Ruby?

What’s the deal? It’s recognition of the fact that building systems is about how you think and reason which requires sound understanding of theory and how to apply it. It doesn’t matter how much code you can write or in what language because delivering a project is about a whole lot more than code.

So often I see companies create job specs for engineers where the key requirement is to hire someone who can hit the deck coding like mad using whatever tools have been selected. To that end they load the specs up with endless tech hubris and at interview ask the details of this or that bit of syntax or API call. But what about the next project within the company where the tech is different? All those engineers that just got hired are now useless, they don’t have the skills and we lose time whilst they learn. Or we could fire them and hire another lot?

Of course what happens more often than not is that companies ensure they don’t use new tech. Instead they force new projects into using all the same stuff they used before. This is a design disaster as now technology is dictating not design or suitability to requirements. A company that follows the hit the deck coding mantra just has deathmarch and no career progression stamped all over it.

Keeping an eye on trends and keeping abreast of new technology is a good thing to do but the larger context of what to use when, when to build rather than buy, when to dump something because it’s warping the design, when to dump one design approach for another (e.g. going from centralized to distributed) and so on is what really matters. This requires thinking, not an encyclopedic knowledge of a huge number of technologies.

Tech is for sissies – Concepts, principles, patterns, measurement, theory and so on are what matters.

[Confession: The title for this entry was inspired by a recent piece from Pat Helland, one of my favourite thinkers]

Technorati Tags: design, philosophy, engineering, software, systems

This scenario get’s played out all the time in IT. The young guns claim that the old guys are out of it, don’t get the latest tech, aren’t smart enough whilst the old dogs smile and are heard to say they’ve seen it all before.

There’s a fundamental tradeoff at work here:

1. Intelligence allows us to at least potentially progress faster
2. Experience allows us to avoid making mistakes as we make progress

Thus a bright inexperienced person may make fast progress but they’re much more likely to make mistakes which will slow them down. In contrast the experienced person may make slower progress but fewer mistakes. Classic hare and tortoise. Who wins?

The nature of software is such that the mistakes we make can take a long time to manifest and when they do, they cost us big time. Thus:

1. Mistakes don’t result in short-term localized damage rather they are far more disruptive with long-term, difficult to clean up damage
2. The time between the root cause of the problem and it’s costly manifestation is large.

It follows that for our intelligence to count we must be able to see sufficiently far ahead to spot our mistakes coming before they get out of hand. Is this achievable? I think software history says it’s not and thus experience is our only tool for understanding root causes and spotting the early signs of an approaching asteroid.

I reckon there’s a lot to be said for the old tradition of master craftsmen handing down their knowledge and experience to apprentices…. (and perhaps the old dogs can learn a few new tricks along the way).

Technorati Tags: process, software, technology

Patrick on the lack of decent reasoning in WS-* world:

“It is precisely this lack of rigor that has failed ws-* from the start”

If this problem were confined to WS-* it’d be a cause for celebration…..sadly it’s pretty much everywhere in software these days.

Update: Was wading through my bloglines feeds and forgot about something Bill said a while back which is relevant.

Intel wades in on the “we can’t do any more magic concurrency for software” issue.

It’s been debated often enough and always seems to come down to the fact that the average programmer isn’t able to cope with concurrency and needs higher levels of abstraction to do it for them. The thing is, we already have such abstractions e.g. transactions and we know they can only take us so far. Worse there are other abstractions out there such as blackboard systems which these average programmers either can’t or won’t try to cope with.

So what is to be done? Well if the last couple of decades are anything to go by, absolutely nothing! Why? It’s the talent limit. How many chip designers are there in the world? How many motherboard designers? How many car designers? How many developers? I’m willing to bet that there are considerably more people in the developer category than any of the others. This is because we’ve lowered the bar in terms of developer quality to cope with the wide demand for bums on seats and lets face it, it’s unlikely that the average enterprise is going to change it’s policies in this respect. It’s interesting to note that it’s much harder to lower the bar in for example chip design, it either works or it doesn’t whilst software is almost expected to be flaky these days.

Until we decide to clean house in software land, we’ll not get progress on these thorny issues because they only matter to the few and when all said and done, there is a school of thought that might suggest that software is good enough, concurrency, efficiency, quality and green’ness be damned.

Update: An example of how challenging it can be to make concurrency easy can be found here. On the surface we’ve done some good things and yet we are still open to the simplest of errors (in this case a missing synchronized clause).