Pragmatic Dictator

I’m currently working on getting 2.0-alpha3 out the door which means all the usual release procedures and in particular a lot of thumb-twiddling whilst various tests run. People often ask about the testing that goes into a Blitz release and so, given that I’m suffering from test-induced thumb-twiddling it seems like this would be a good time to write something down.

Blitz is basically a layered Jini service with remote wrappers cleanly separated from the core JavaSpace implementation. Thus we can thrash the core without needing to do a full remote deployment which provides many benefits such as eliminating a lot of network I/O that can hide race conditions and deadlocks.

My test machine is a dual processor dual core machine (2 x 2.6Ghz Xeons with HyperThreading, 2Gb, 2x70Gb SCSI Disks) and it’s main duties are to run the long term soak tests for long periods of time (unit tests get run as part of day-to-day development on my PowerMac Dual G5). There are two basic types of soak test that get run regularly they are:

1. TxnStress – a multi-threaded test which fills a Blitz instance with a fixed number of entry’s and then randomly takes and re-writes one of those Entry’s with each take/write pair done under a separate transaction.
2. Stream – a multi-threaded test which writes a sequence of Entry’s into Blitz whilst another thread takes them. Writers and Takers are run in pairs up to a configured number.

Whilst these tests run I typically have jconsole hooked up plotting memory consumption so leaks can be detected and fixed. In addition, Blitz’s in-built statistics API is used to check queue sizes, number of entry’s etc (this is the same API used by the remote dashboard). Finally, each test has code that ensures we are not leaking Entry’s or missing them. Blitz also has a debug-mode where one can simply connect to a pre-configured socket and trigger the dumping of statistics to the console at any time.

These tests get run against Blitz in both transient and persistent mode (which is a mere flip of a configuration variable).

Stream is just about done, so I’ll be shipping a release to SourceForge shortly.

Technorati Tags: javaSpaces, jini, Blitz

Dave has some thoughts on the recent W3C “Web Of Services For Enterprise Computing” get-together.

He notes:

“Given all the angst about Web architecture vs Web services architecture, I was also surprised that there was no support for technical reconciliation. I suggested WADL (Web application description language), to help with enterprises and the desperate perl/python hacker building stronger typed REST services. And for the flipside, I suggested improved SOAP to URI/XML bindings so SOAP/WSDL services would be more easily consumable by REST clients.”

No support for technical reconciliation? Why would either side want technical reconciliation when each believes it’s approach is “right” for it’s own needs?

I think he hits the nail with a comment earlier in the posting:

“There was almost no discussion about how enterprise computing is different from “non-enterprise computing””

Which is sad but not surprising – a good amount of the reason we have the whole Web/WS-* debacle is down to the fact that this kind of discussion didn’t happen when it should have. And if we still aren’t having the discussion, technical reconciliation is likely to be impeded.

Technorati Tags: distributed systems, enterprise, web

Apparently not Anne:

“…..But I’m not so good at blending into the traditional, primarily because I’m just not interested in doing that.

A further issue is that though I have the background to comment on enterprise software, it doesn’t excite me anymore. And every minute I spend looking at enterprise software is a minute I’m not looking at the latest in social media and social tools and new social forms on the web, my real area of interest…..”

For all that enterprise is big business and pays the bills, in many cases it’s not all that interesting. Once you’ve done a couple of integration jobs, you’ve done them all.

All of which leaves me wondering:

1. Might there be some connection between repeated reinventions of the wheel and the run-of-the-mill sameness of most enterprise jobs?
2. How much talent can one really expect to attract into enterprise?

Technorati Tags: enterprise, technology, web

In the last couple of years we’ve seen the arrival of the mashup which is at least on some level nothing more than the latest in a long line of terms for integration. Thus far most mashups consist of a simple amalgamation of a couple of services which leads to a very flat graph of service dependencies. Service dependencies are things like:

1. Data Schemas – structure of data provided by services
2. Endpoints – location of service be it a URL or a WSDL endpoint
3. Availability – whether a service is available for use
4. Reliability – whether a service that is available behaves as expected

These mashups are already at the mercy of the underlying services they are built on. The mashup provider has little control over these services. For the most part these mashups work but it’s because they have only a few moving parts such that the likelihood of issues is low.

Many enterprises have considerably deeper dependency graphs inside their firewalls and have to work hard to keep them stable. There’s probably some limit to what can be achieved once the dependency graph gets beyond a certain depth and it might well be that the maximum depth is smaller once external services are brought into the mix. The maximum depth is likely further reduced because these enterprises wish to treat external services as if they are part of their organization. They want to be able to integrate them using transactions, they want them to have the same level of reliability as what lives within their own data-centres, they want integrated security options etc.

I suspect that a lot of what can be done in a single enterprise (at substantial cost) such as high reliability is going to be considerably more (prohibitively?) complex to achieve across organizations. This is because the level of control required to achieve these targets is beyond that available across enterprises.

Right now I think there’s much effort being made to paper over these issues such as features in WS-*, SLA’s etc. I wonder if it might it be better to give up on this idea of control and build some simpler solutions….

Technorati Tags: distributed systems, enterprise, utility computing, web

I’m not talking personal hygiene which of course is important (your Mum is right, brush your teeth, bathe regularly etc) I’m talking protocols.

Pete’s highlighting the issue which comes down to this strange comment from Paul.

SOAP isn’t RPC? Well sure if I read this I can see the statement Paul references:

“SOAP is fundamentally a stateless, one-way message exchange paradigm, but applications can create more complex interaction patterns (e.g., request/response, request/multiple responses, etc.) by combining such one-way exchanges with features provided by an underlying protocol and/or application-specific information.”

Trouble is the same document also says:

“[SOAP Part2] defines a data model for SOAP, a particular encoding scheme for data types which may be used for conveying remote procedure calls (RPC), as well as one concrete realization of the underlying protocol binding framework defined in [SOAP Part1].”

And then (here):

“One of the design goals of SOAP Version 1.2 is to encapsulate remote procedure call functionality using the extensibility and flexibility of XML. SOAP Part 2 section 4 has defined a uniform representation for RPC invocations and responses carried in SOAP messages.”

What does [SOAP Part2] say?

“One of the design goals of SOAP is to facilitate the exchange of messages that map conveniently to definitions and invocations of method and procedure calls in commonly used programming languages. For that purpose, this section defines a uniform representation of remote procedure call (RPC) requests and responses.”

If we go by the letter of the base-spec SOAP isn’t about RPC but if we look at the spirit, it’s stated that a design goal for SOAP is to support RPC. Note the number of times RPC is mentioned in the base and adjunct specs. Seems like RPC is a fairly fundamental part of this whole thing. Confused?

Update: Don Box wrote up this brief history of SOAP which also makes the link to RPC

Technorati Tags: distributed systems, SOAP, technology, web