The debate about all these many core processors continues to circle the blogosphere. Tim Bray had this to say which set me thinking (always a bad thing):

Any time we have a piece of state that needs to be accessed concurrently we hit problems. One can hide this problem using messaging (or similar) but the key aspect in these solutions is that we can partition operations into streams against discrete elements of data (a discrete element could be a group of things) that don’t interfere with each other. Partitioning however can be problematic:

1. Our data has to be amenable to partitioning via hashing or some other method.
2. It gets tricky when we need to deal with availability and disaster recovery.
3. Getting the correct granularity of partitioning can be challenging.

Which is interesting because whilst we’ve eliminated the concurrency issue, we’re now faced with a different one (partitioning) which could be just as hard to cope with and requires just as much thought from a developer and/or architect. Coincidentally, Werner Vogels (Amazon) is going to be talking about an internal data store (HASS) at the Google Scability Conference and specifically the problems of partitioning and consistent hashing (my original interest with respect to this talk was in the context of the CAP conjecture).

Another means of avoiding all these concurrency issues is to push them somewhere else. More often than not this becomes an exercise in creating a supposedly stateless system which in reality simply puts all the state in one place, usually the database. The argument is that this is acceptable because it’s only the likes of databases that should deal with these hard issues.

The rub with having the database handle it is that the concurrency model it uses will only scale across so many processors (more if you’re read mostly, less if your not) and cope with so many concurrent accesses from the stateless component. Once again to get our database layer to scale, we’ll need to partition our data into shards across multiple databases (an approach adopted by a number of top-line websites) or find some other way to reduce concurrent load on the database instance.

The act of partitioning can mean we reach a point where we can no longer expect to have atomic updates because the mechanisms for achieving it (e.g. two-phase commit) stop us scaling. When this happens we must construct complex or at least exotic solutions such as that proposed by Pat Helland.

Okay we got rid of our concurrency problem and swapped it for a partitioning problem which then turned into something of an exotic problem. Are we any better off? It seems no matter which way we go we end up with some tough problems to solve.

Perhaps there’s a sweet-spot tradeoff where the combination of a CMT box, with data partitioned across a number of processes and each process containing a simple concurrency model covers most situations. Even if that’s the case it seems developers will have to learn a few new tricks.

Technorati Tags: architecture, concurrency, software, systems

Update: A good comment over on Reddit.

Woohoo, after jumping through some hoops I’m going to be able to attend the conference. I will of course be representing Betfair.

I’d be happy to attend any of the talks but my personal favourite is the storage talk by Swami Sivasubramanian and Werner Vogels. I’m really looking forward to making some new contacts. If anyone can recommend some good Seattle beers to try, drop me a line.

Patrick on the lack of decent reasoning in WS-* world:

“It is precisely this lack of rigor that has failed ws-* from the start”

If this problem were confined to WS-* it’d be a cause for celebration…..sadly it’s pretty much everywhere in software these days.

Update: Was wading through my bloglines feeds and forgot about something Bill said a while back which is relevant.

Kind of - see Juan Martin should never really rock given he’s a flamenco guitarist. By any measure he’s a damn fine guitar player and along with his Flamenco Ensemble consisting of:

• Two dancers - one guy, one gal, amazing balance and posture, fast feet with tremendous fluidity
• One singer - with a scary big voice and a serious sense of rhythm conveyed via foot stamping and clapping (goes for the dancers too by the way)
• One flute player - also pretty mean with sax and clarinet
• One percussionist - I swear this guy had more than two hands

….put on a great show, Friday night in the Reading Concert Hall. An impressive display with the simplest of musical instruments (bongo drums and even a wooden box). And the ticket price - less than twenty pounds.

If you get a chance, go see them.

Technorati Tags: music, personal, Juan Martin

It’s the end of my working day, time to head for the train station and make my way home. Because I’ve done this trip many a time it’s a fairly well honed process:

1. I have a rough idea of how long each leg of my journey takes so if I need to be home for a given time, I can backtrack from there and figure out roughly when I need to get a train.
2. I leave work on most days at about the same time so I know which trains are when, how many stops there are and which will stop at Reading.

But there are days when this doesn’t work because the trains aren’t running to time. So I fall back to a much simpler approach which is to:

1. Look at each train departure
2. See if it’s going to Reading
3. See if it is making fewer stops than my current choice
4. See how soon it’s leaving

Based on the above I make a guess as to which train is best and climb aboard. Of course, the train I’m on can break down in which case I’ll be dumped at some intermediate station and the process starts again.

In these modern times things are made easier because departures, destinations and so on are announced and displayed on big electronic scoreboards but that’s an optimization. My process doesn’t require much in the way of extra information beyond what I could get from the driver and a map of the available stations and routes (often some subset information is posted at individual stations which means I don’t need to carry all this information around in my head).

Notice also how I don’t actually need a sense of time because in the worst cases I can dump my needs for predicting/controlling my arrival time and just take the first train I can find that gets me towards my destination.

Thankfully most trains do run to time but on any particular day, some don’t. Sometimes there’s an announcement that tells me why the train hasn’t arrived, whether it’s late or cancelled and I can then decide to wait or make other arrangements. What if there isn’t an announcement? Well I’ll wait a while and then assume the train is not going to arrive. Whether I get an announcement or not I can still make progress by virtue of my self-imposed waiting limit. Of course I might miss an announcement because I didn’t wait long enough but it doesn’t matter, I’ll still find a train. Even in the worst case where I miss the last train home, I can curl up somewhere and wait until the first useful train the next day.

You probably aren’t worried about my daily journey or whether I make it home but you might want to think about the above in the context of polling, timeouts and events. And you might want to consider that only polling and timeouts are really necessary for me to find a train to get me home. Event’s help me optimize but aren’t necessary. And you may notice that I require only a minimum amount of knowledge and it’s usually available locally from train driver or station wall.

Distributed fault tolerant systems are everywhere and seem only to need the simplest of underpinning mechanisms to make them work.

Technorati Tags: architecture, availability, distributed systems, software