Pragmatic Dictator

There’s been a lot of stuff going on of late, I certainly feel brighter than I have done in some time. It just takes time to work through things, gain perspective and come to conclusions.

But some things never really get “closed out”. Some things you look back at and wonder if they really were as good as you thought at the time. Were they really worth the effort or did they cost you dearly in subtle ways you didn’t notice? Were they real or simply a function of over-optimism, daring to dream and a little too much rose tinting of shades? Would I have been better off if it had never been?

“What if?”, “why did I bother?”, “was it worth it?”, “why can’t I let it go?”.

Are such things important? Who knows?

Technorati Tags: philosophy, personal

I attended Sun’s London Tech Days conference yesterday. I came away with a couple of overwhelming impressions.

We have serious problems in terms of the quality of our developers. Some of the questions being asked should be reserved for those getting to know the trade at school or university and not be uttered by “paid professionals”. Simply horrifying……and here’s some further food for thought.

Sun have this saying “The Network is The Computer” – based on yesterday I’d say that’s true for all the wrong reasons. See what they really mean is “The Network is One Computer”. There was lots of talk about multiple cores, multiple threads (no mention of the likes of SEDA), race condition detectors (a cool piece of tech by the way), optimizing compilers and so on.

But there wasn’t a single mention of what they would do when you grew beyond a single machine. The closest they came was a few brief mentionings of clustering which only counts for tightly-coupled, uniform computational scenarios. All those web 2.0 sites, Amazon, Google and so on are building systems on top of multiple boxes which need to work together in ad-hoc, changing configurations.

In our current systems construction doctrine we still focus on building our application inside of a single machine out of bits (e.g. Spring or App Server style). Witness how we strive to allow developers to run all that’s required on their own machine and rarely force them to run remotely. Do we really feel this is a good thing given that when the code is given to ops the first thing they do is put it on lots of machines? What incentive does a developer have to write monitoring tools to help ops out when they don’t see the pain in development?

Standard development practices are not aligned with what goes on in deployment, Sun and others need to start bridging that gap and developers need to start working in a style that fits much better with what goes on “over the wall”. Certainly it’s not going to be easy given we have such a long legacy of confining ourselves to a single machine but it’s necessary.

The fact of the matter is we know developers don’t like threads, transactional memory might be a nicer model but it won’t scale any better and virtualization makes it easier to spread software across multiple “machines”. The future looks like it will be multiple, co-operating, separate (possibly remote) processes – now where are the tools etc to make that happen?

Technorati Tags: distributed systems, engineering, sun, technology

Having worked as an independent contractor for a number of years, I’ve finally been tempted to take a permanent job by Betfair. My decision to join them was driven by several factors:

1. The size of the challenge – Betfair’s systems are big, getting bigger and doing serious numbers
2. Ability to influence – as a contractor you are often pulled in to fire-fight or work within what has already been done you don’t get to innovate or change
3. The tech team – I gotta get more of these guys to blog. For now, I’ll point you to Ewan Silver
4. The kind of challenge – it’s broad and not about Jini

Those of you who know my technical background might be able to guess at the sort of things I will be doing. I hope to provide more specifics over time but for now, speculate away!

Some of you may be wondering what this might mean for Apache River and/or Blitz. I think this change will benefit both projects:

1. Blitz will return to being a fun side project with I believe an attendant rise in my enthusiasm – which you can already see in 2.0alpha
2. Apache River could really do with real-world experience such as that which might be gained from working in the Betfair environment

All that remains for me to do is thank all those who have supported me one way or another during my “Lone Crusader” years. That would include Vertex, Tangosol, various at Sun Microsystems (including alumni) and a too-long-to-mention list of individuals – good relationships I plan to maintain.

Technorati Tags: betfair, personal, technology

Ever had the feeling you just weren’t the kind of person someone had in mind when they designed a form?

A couple of weeks back I was filling in such a form which was asking for details of all the companies I’ve worked for over the last 5 or so years. In addition, they wanted contact details for my line managers at those places of work.

Hmmm, you kind of know they want to get references from these places which is difficult when your current company is your own and the previous one was joint-founded by you and others. I just knew as I wrote the details down it wasn’t going to work. Sure enough…..

“Could we please have contact details for business referees that can confirm Mr Creswell’s involvement with Lone Crusader Ltd“.

There now, wouldn’t it just have been eaasier to do what everyone else does and ask for referees rather than last places of work and line management?

Being outside the demographic is sometimes a compliment and sometimes a curse…….

Technorati Tags: humour

How often do we admit something’s wrong and take corrective action? How often do we admit something’s wrong before it gets well out of hand?

There are two basic things we need before we can admit something’s wrong and address the problem:

1. Be strong enough to admit there’s something wrong
2. Be aware that something is wrong

The first item is about having sufficient strength of character to admit our mistakes, learn and move on. Inhibitors for this include ego or pride, fear and external environmental factors (e.g. a violent spouse).

The second item is about recognizing the indicators of a problem. For us humans indicators can be anger, frustration, lack of drive, fatigue, health problems or a constant need to distract oneself via activity. In terms of computer systems, indicators might be Java exceptions under odd circumstances, strange lockups, weird performance issues etc.

The art whether in the human or systems world is detecting these problems early on and reacting to them before they get out of control. In the systems world, so many times the first we know of a problem is when we see an exception trace or the pager beeps at 2am but is this really necessary?

Could we not have been aware of the problem earlier? We put a lot of effort into testing but considerably less effort into endowing our systems with monitoring and feedback mechanisms that can provide us with useful statistics. These statistics are the things that can give us prior warning of a problem and come in many forms:

1. Too many tasks
2. Too many threads
3. Queues that are too long
4. Time per task increasing

Frequently we build logging into our software but this is really a postmortem facility something that is only useful after the problem has occurred. Our OS’en, hardware and software platforms often have monitoring infrastructure built-in, why is it that we so often choose to build our applications without similar facilities?

Technorati Tags: monitoring, performance, systems health