Pragmatic Dictator

There’s been a lot of stuff going on of late, I certainly feel brighter than I have done in some time. It just takes time to work through things, gain perspective and come to conclusions.

But some things never really get “closed out”. Some things you look back at and wonder if they really were as good as you thought at the time. Were they really worth the effort or did they cost you dearly in subtle ways you didn’t notice? Were they real or simply a function of over-optimism, daring to dream and a little too much rose tinting of shades? Would I have been better off if it had never been?

“What if?”, “why did I bother?”, “was it worth it?”, “why can’t I let it go?”.

Are such things important? Who knows?

Technorati Tags: philosophy, personal

I attended Sun’s London Tech Days conference yesterday. I came away with a couple of overwhelming impressions.

We have serious problems in terms of the quality of our developers. Some of the questions being asked should be reserved for those getting to know the trade at school or university and not be uttered by “paid professionals”. Simply horrifying……and here’s some further food for thought.

Sun have this saying “The Network is The Computer” – based on yesterday I’d say that’s true for all the wrong reasons. See what they really mean is “The Network is One Computer”. There was lots of talk about multiple cores, multiple threads (no mention of the likes of SEDA), race condition detectors (a cool piece of tech by the way), optimizing compilers and so on.

But there wasn’t a single mention of what they would do when you grew beyond a single machine. The closest they came was a few brief mentionings of clustering which only counts for tightly-coupled, uniform computational scenarios. All those web 2.0 sites, Amazon, Google and so on are building systems on top of multiple boxes which need to work together in ad-hoc, changing configurations.

In our current systems construction doctrine we still focus on building our application inside of a single machine out of bits (e.g. Spring or App Server style). Witness how we strive to allow developers to run all that’s required on their own machine and rarely force them to run remotely. Do we really feel this is a good thing given that when the code is given to ops the first thing they do is put it on lots of machines? What incentive does a developer have to write monitoring tools to help ops out when they don’t see the pain in development?

Standard development practices are not aligned with what goes on in deployment, Sun and others need to start bridging that gap and developers need to start working in a style that fits much better with what goes on “over the wall”. Certainly it’s not going to be easy given we have such a long legacy of confining ourselves to a single machine but it’s necessary.

The fact of the matter is we know developers don’t like threads, transactional memory might be a nicer model but it won’t scale any better and virtualization makes it easier to spread software across multiple “machines”. The future looks like it will be multiple, co-operating, separate (possibly remote) processes – now where are the tools etc to make that happen?

Technorati Tags: distributed systems, engineering, sun, technology

Having worked as an independent contractor for a number of years, I’ve finally been tempted to take a permanent job by Betfair. My decision to join them was driven by several factors:

1. The size of the challenge – Betfair’s systems are big, getting bigger and doing serious numbers
2. Ability to influence – as a contractor you are often pulled in to fire-fight or work within what has already been done you don’t get to innovate or change
3. The tech team – I gotta get more of these guys to blog. For now, I’ll point you to Ewan Silver
4. The kind of challenge – it’s broad and not about Jini

Those of you who know my technical background might be able to guess at the sort of things I will be doing. I hope to provide more specifics over time but for now, speculate away!

Some of you may be wondering what this might mean for Apache River and/or Blitz. I think this change will benefit both projects:

1. Blitz will return to being a fun side project with I believe an attendant rise in my enthusiasm – which you can already see in 2.0alpha
2. Apache River could really do with real-world experience such as that which might be gained from working in the Betfair environment

All that remains for me to do is thank all those who have supported me one way or another during my “Lone Crusader” years. That would include Vertex, Tangosol, various at Sun Microsystems (including alumni) and a too-long-to-mention list of individuals – good relationships I plan to maintain.

Technorati Tags: betfair, personal, technology