Pragmatic Dictator

Jonathan Ive (Apple’s Design High Priest) has been knighted. Those that succeed in influencing technology as much as Mr Ive are well worth paying attention to.

A quick google, shows that he’s not that public of a personality, seemingly preferring to let his products do the talking, works for me! Nevertheless there are some interviews out there, including this one which features a quote that I feel fits well (and is far more succinct) with my previous post:

Q. What are the advantages of designing for one company? And the disadvantages? What are the particular characteristics of the set-up at Apple that has made the experience of working there rewarding for you?

A. It is pretty humbling when so much of your effectiveness is defined by context. Not only is it critical that the leadership of a company clearly understands its products and the role of design, but that the development, marketing and sales teams are also equally committed to the same goals. More than ever I am aware that what we have achieved with design is massively reliant on the commitment of lots of different teams to solve the same problems and on their sharing the same goals. I like being part of something that is bigger than design. There is a loyalty that I have for Apple and a belief that this company has an impact beyond design which feels important. I also have a sense of being accountable as we really live, sometimes pretty painfully with the consequences of what we do.

I’m a CTO, the traitorous techie that turned to the dark side of management or the unrealistic techie that won’t face up to management reality, depending on your point of view. Who am I really? I’m the man with experience on both sides of the fence. I’m the one who sees that the grass is decidedly scorched regardless of which side of that fence you sit.

1. Many geeks are obsessed with using all the latest technologies and building ultra-clever software that isn’t practical to run assuming it ever gets delivered.
2. Many managers fail to prioritise. An organisation can only ever be focused on one thing effectively at any moment. That’s because there are always bottlenecks, cash, mindset, staff, office space, the list goes on. Best identify the single most important thing you can do and go for it with all you have. Back a single horse, not the entire field.
3. Many geeks are all about the process. They’re agile, they’re kanban, they’re lean, they’re waterfall, they’re hypothesis testing. All of them miss the fundamental, which is not the process but the mindset that underpins it. Following the process blindly makes one a valueless robot, a cog in the machine, rather than an active participant shaping destiny.
4. Many managers fail to understand that dates are aspirational. One must actively work towards a date by managing scope, risk and resource, even then, there are no guarantees. What’s the value of a date anyway? Nothing. What matters is what you have to offer and whether it’s enough.
5. Many geeks fail to understand that design is a creative, subjective discipline. Design patterns aren’t the answer nor is the do it all framework. It’s knowing what to apply and when which comes from broad experience (which means you need to know more than one language, platform and operating system and will need to have worked in a number of industry verticals).
6. Many managers are obsessed with cost reduction. Fact is that to do something to a particular standard in some reasonable amount of time is going to cost a certain minimum. That minimum is largely unknowable in advance so better to decide how much you are prepared to pay and work to that number. As soon as an overshoot looks probable it’s time to step back and re-think (maybe even can) the plan.
7. Many geeks are performance junkies, optimising every part of their system based on a set of metrics that have nothing to do with the real-world. In the worst cases, there are no metrics, no one has done any measurement and the statement that something is “too slow” is pure speculation.
8. Many managers treat those that work for them as interchangeable components with no feelings, no individual strengths or weaknesses, no motivations or concerns. They then wonder why staff retention is appalling, performance is horrid and they can’t ever seem to recruit the right people.

What’s missing is a collective focus on producing something worth a damn. Producing such a thing requires caring and provides challenge, profit, happiness, satisfaction and a whole lot more. So the first question to ask whether you’re a geek or a manager is, what are we trying to do? Natural follow-ups would be why and do we care?

Whilst you’re pondering that, hopefully you’ll realise that there shouldn’t be a fence or sides…

When I arrived at Sporting Index, three or so years ago, my early tasks included the planning of a programme of work to overhaul the existing trading system.

To call it a trading system was, at least architecturally, a gross lie, as in fact it was an everything system: payments, accounts, customer profile, reporting (yes, OLAP and OLTP on the same database, madness) and the bet engine.

So the programme broke down into two parts:

• Clean up and separate out the components
• Replace certain components with new implementations

The programme of work started about 18 months ago and if we deliver the entire roadmap there’s another 18 months to go.

Thus far we’ve separated the B2B elements out (yes, they were hanging off the side of the “trading system” as well) and put a new data delivery infrastructure in place with considerably reduced latency and increased reliability. We’ve also just about completed the moving of all reporting into our OLAP systems with real-time updates from the OLTP elements (we used to do reporting refresh every 24 hours with all the painful load spike issues that go with that). The other essential element has been to eliminate the intimate relationship between website and trading system (most website content should not live in the betting engine).

The next major step we’re focused on is the splitting out of customer and account handling. Once this is done we’ll be in the happy situation where we can introduce our new bet engine and run in parallel with the old one so a customer placing bets on markets in either engine continues to get a complete and accurate view of their position (as do our traders).

Our other major area of focus is the development of a new betting engine and a key innovation there will be that we don’t use RDBMS’en for storing that information. We maintain auditing trails and DR abilities but with a faster, far lighter weight solution that will cost much less than what we’re running now.

Some technical details:

• We’ve opted for a service-based implementation, mostly with RESTful interfaces and always with smart stubs. Fact is we have to do a distributed solution to support our regulatory requirements effectively and efficiently (PCI, FSA and various gambling authority needs).
• We’ve implemented a service lookup mechanism from scratch based on gossip algorithms. This allows us sophisticated load and failure management strategies tuned on a service by service basis. It also gives us scope for admission control.
• We’re building up a new multicast infrastructure to deliver updates from the bet engine to desktops, other systems etc in real-time.
• Our bet engine is partitioned such that we can up or down scale on demand via virtualisation (no we can’t use most forms of cloud infrastructure as that breaches a number of regulations).
• We’ve got some nice automated recovery protocols that make recovery from hardware or component failures straightforwards for operational staff. In essence, they replace the broken element and it automatically knits itself back into the system and supports an SLA for recovery. For example, we can say that a cache will contain all relevant data within 5 minutes assuming a certain set of constraints are met (failure recovery times are difficult to guarantee 100%).
• Everything is monitored including stubs, services and infrastructure. Our operational teams get to routinely use what’s being developed and be involved in the specification of the data generated and the writing of the manuals. We’ve standardised the protocols/methods of exposure for both the monitoring data and logging output.

You’ll notice I haven’t talked about languages used and such. That’s because it doesn’t matter as with our service-based approach we can use whatever suits us best on a per service basis. That’s a key part of our general engineering philosophy, “right tool for the right job”, we don’t do fashion, buzz or hype influenced work, the pragmatic, practical, effective and efficient space is where we’re focused.

Many in IT would have you believe that they’re progressive, constantly advancing and learning. They will provide examples such as:

• We’re adopting lean (or agile)
• We’re using Scala
• We’re doing large-scale

Dig a little deeper though and you’ll quickly find:

• The agile or lean they’ve adopted and trumpet is merely a fixed set of ceremonies such as time-boxed cycles, limiting in-flight work, pair programming and automated testing.
• The Scala code they’re writing is largely imperative with no use of any of the obvious functional aspects.
• The supposed large-scale is in fact a couple of thousand customers running on a single database with an operation count per minute of no more than 60.

These individuals and companies are paying lip-service, they haven’t learnt anything other than what the latest buzz is and they’re operating some parody of the real thing.

The fundamental problem is, they don’t do their research. They don’t actually examine what has been done previously and understand its fundamentals, apply it and sharpen it. This is the skill that I gained at university, not some set of buzz technologies or processes. This is the fundamental skill that allows us to progress, to learn, adapt and act.

[ There's a supreme irony in an individual or company claiming to be doing agile or lean and not being able to research, learn and adapt because they follow a fixed set of ceremonies. ]

Unfortunately this skill isn’t generally recognised as valuable in the IT industry, it’s rarely taught and not seen in requests for curriculum changes from business to universities. This sort of feedback is more typical:

…startups need graduates who can hit the ground running, who are proficient in PHP, Python and Ruby (among other modern programming languages), and who, ultimately, understand the practical side of software engineering as opposed to just the theoretical side which they learn at university.

Being proficient in a language does not make you good, it just means you can crank (poor) code fast. Further, understanding the practical side of software engineering means learning from what you do, analysing and adjusting. If we’re all so good at that, why do we repeatedly get burnt by the same classic mistakes?

Far too many within IT act on hype or pay lip-service, they don’t do research, they don’t adopt the basic disciplines of learning. Whilst that continues, progress is nothing but a pretence.

As soon as we give something a name, it becomes open to abuse and misuse.

Vendors can claim they are doing it and support it, developers can claim they do it, use it or implement it. There are a bunch of ready examples: Agile, XP, SOA and REST. Naming something makes it easy to ignore or forget its underpinnings, the elements that deliver value.

As a martial artist, I’m familiar with this pattern of behaviour: various people claim to practice and teach authentic Silat, Karate, Kung Fu, Escrima and so on. Inevitably some of them are exposed as pretenders. One of the more notable martial artists, Bruce Lee was sufficiently concerned about this that he gave serious consideration to leaving his approach to martial art (Jeet Kune Do) unnamed^*.

Is it worth naming things? Might we be better served by making our knowledge, approaches and philosophies visible for others without naming them to adopt or not as they see fit? Would it reduce the number of valueless certifications, buzzword cv’s and endless wars over which way is the way and who’s doing it right?

* Jeet Kune Do (1997) ‘Actually, I never wanted to give a name to the kind of Chinese gung fu that I have invented, but for convenience sake, I still call it “Jeet Kune Do”. However, I want to emphasize that there is no distinction between jeet kune do and any other kind of gung fu, for I strongly object to formality, and to the idea of distinction of branches.’