Met Kyle a few years back on a consulting gig down in Nashville. Been nagging him not quite ever since to blog and finally he’s seen sense.

Should be good….

Technorati Tags: blog, technology

Everybody’s pitch is about the great things that can be done with their technology, method, architectural approach etc. They’ll sing it from the roof tops, put it up on websites, spam us with email and so on.

But they rarely (if ever) discuss the flip-side - what their stuff is not good for. Sometimes:

• selecting an appropriate solution is more about what something is bad at than good at
• the easiest way to understand something is to know it in terms of what it can’t do

It’s amazing how often asking someone about the negatives of their stuff results in silence.

Technorati Tags: architecture, design, systems

One designs or architects solutions to problems - technologies be they ESB’s, CORBA etc are merely implementations of (parts of) an architectural solution. Note that one might compromise one’s architecture slightly because it’s expedient to use an off-the-shelf product. Regardless the important point is that architecture comes first, not last and yet that’s rarely the case these days. Instead technology rules…..

Consider how prevalent the use of frameworks is within our industry and think about the fact that in many cases one simply writes a POJO or two and leaves the rest to the framework. The framework makes life easier, it solves the big problems but it also exerts force on the design of our software as after all we must write it to follow the appropriate conventions, implement the appropriate methods etc.

The very worst example of the framework trend is seen in the decision to purchase a mammoth framework offering that provides everything in one box as an “integrated solution”. A huge stack that get’s connected into everything and exerts massive gravity on our architecture. Everything becomes an exercise in warping aspects of our system to fit with this stack and the assumptions of its creators. Essentially we’ve bought “architecture in a box”.

We’ve been doing this sort of thing for a long time and there’s even a business case to go with it. Enterprises want commodity developers. These chaps are not trusted to take on the bigger challenges rather it’s deemed appropriate to use frameworks (in the form of middleware) to address these big issues and confine the developers to the task of simply implementing the business logic.

There’s an entire industry of analysts and others devoted to producing endless tech comparisons to determine just which of the myriad of frameworks will be the single, final silver bullet solution that allows one to implement an entire system in a matter of weeks. This stuff is gobbled up by the the commodity programmer brigade.

Changing these behaviours is challenging: Architecture RIP? Possibly but a recent comment on ESB’s from Ron Schmelzer means there’s still hope. And there is at least one part of the developer universe that sees value in architecture.

Technorati Tags: architecture, design, technology

Never let ignorance stop you from having an opinion.

Never let an ignorant opinion stop you.

[ via Greg Guerin on the OS X/Java list ]

The release of the Dynamo paper has generated a lot of interest around the net. That’s more than appropriate because I don’t think there can be any doubt that Dynamo is a great piece of work.

It seems there might be a further bonus that’s largely gone unmentioned (even Greg seems to have missed it) but has been hinted at by Werner at various points in the past. Read carefully and you’ll find some details of a custom invocation infrastructure:

“Both get and put operations are invoked using Amazon’s infrastructure-specific request processing framework over HTTP. There are two strategies that a client can use to select a node: (1) route its request through a generic load balancer that will select a node based on load information, or (2) use a partition-aware client library that routes requests directly to the appropriate coordinator nodes. The advantage of the first approach is that the client does not have to link any code specific to Dynamo in its application, whereas the second strategy can achieve lower latency because it skips a potential forwarding step.”

Notice how they have support for both smart and dumb clients with the smart client setup being somewhat akin to a pattern that’s been seen in Google’s software including Chubby. The choice to reuse http would give them an option to leverage many a load balancer’s capability to apply custom routing by URL which would assist in service invocation routing.

Other interesting tidbits include:

1. A mention at Google Scalability Conference of a lightweight rendering engine that might invoke upwards of 150 requests per page. Given some of the latencies discussed in the dynamo paper I am wondering if this custom framework might have some support for making collections of requests in parallel.
2. Common service types are stateless aggregator services that can perform a lot of caching (wondering how much the use of http helps here) or stateful services.
3. A statement from a past interview with Vogels:

   “The first category is the services that make up the Amazon platform. There we use interface specifications such as WSDL but we use optimized transport and marshalling technology to ensure efficient use of CPU and network resources.”.

   See the mention of the custom framework again but also a possible hint that they make use of a variety of interface specifications (perhaps including something homebrew).

Food for thought?

Technorati Tags: distributed systems, performance, amazon, scalability

It’s well known that abstracting away network failure in inter-process communication is a bad thing. but there are other similarly harmful abstractions one might adopt when handling networks such as assuming uniformity.

In recent times there’s been a resurgence of interest in using messaging between processes as a mechanism for taming concurrency rather than the (possibly) more conventional approach of using threads and locking. This model is very appealing in it’s simplicity and some variations even allow for process failure (though I think there’s still some interesting discussion to be had around being certain that a process has failed rather than become partitioned away by network failure - split brain scenarios etc).

Some are wondering if this messaging approach could be extended beyond concurrent programming across multiple cores in a single box to deal with concurrent programming across networked machines. I think there’s maybe a small fly in the ointment - latency. If all processes are communicating via messaging inside of a single SMP box we will likely have at least approximately uniform latency between processes which is reasonably easy to manage. The same cannot be said of messaging across processes in a NUMA system or on a network. Things get still more tricky if one has processes running on a mix of SMP and NUMA machines all living on a network and messaging each other.

Managing such a mix is difficult - one must consider carefully where to deploy things and the nature of the messages you send (what you’d consider moving around an SMP system’s bus is probably not the same sort of payload you’d want to place on a network). When a process fails, one potentially cannot start up a replacement anywhere rather it must be placed carefully and appropriately.

Technorati Tags: concurrency, distributed systems, messaging, networks