Premature optimisation vs just doing it right

I often hear developers in the tech community quoting the XP slogan 'You Aren't Gonna Need It' (aka YAGNI). 'Why are you doing that? You're optimising too early,' they say, bringing in a reference to premature optimisation as well.

I've seen some particularly heated arguments in the Rails and Java communities around this recently. It's also been the pretext to some of the most frustrating conversations I've had, during my career.

I take to heart the principles behind YAGNI and premature optimisation. There's been many occasions where I've reflected on my own work, and realised I didn't pick the simplest solution or I wasted time on something that wasn't required. In these cases, I would have loved someone to have challenged me with those quotes, it would have been perfect timing.

I also take to heart the mantra that 'quality is free.' This is summed up in a classic book by Phillip B Crosby called 'Quality is free: the art of making quality certain'. The book describes how improving quality can actually improve a business' profitability, by reducing the the costs associated with poor quality and preventing defects. A key message from the book that stays with me is 'focus on improving quality' and other measurements will improve too.

One of the ways I act on the 'quality is free' mindset is by continuously trying to improve the code I write, and never being satisfied with 'hacky', unclear, or poorly designed solutions that are likely to cause pain in the future. It's not just about what you do, but how you do it, a Software Craftsman might say.

Where my frustration comes in is when quality improvements are met with the response 'You Aren't Gonna Need It', or 'you're optimising too early.' This, IMO, is a fundamental misunderstanding of what premature optimisation and YAGNI are about.

To quote Ron Jeffries book, 'Extreme Programming Installed':

YAGNI: "You Aren't Gonna Need It." This slogan, one of XP's most famous and controversial, reminds us always to work on the story we have, not something we think we're going to need. Even if we know we're going to need it.
What you won't find here is mention of doing things badly, and only improving them when it hurts you. In fact elsewhere in the book Ron describes how defects can hurt a project, as well as the importance of coding standards and clear, intention-revealing code amongst other things.

How about Donald Knuth's discussions of premature optimisation?

That also doesn't support this view, if you take the time to understand it. One of the key points Donald Knuth makes is, in fact, premature performance optimisations reduce the clarity of code, introduce unnecessary complications and increase the chance of incorrectness. His focus is on avoiding 'micro-optimisations', whose impact is likely negligible anyway.

The idea that these quotes should be used to justify not refactoring or tidying up code seems completely counter to their intention.

Embracing YAGNI and avoiding premature optimisations doesn't mean you should approach software development in a completely reactive way, not giving any consideration to risk or impact until things go wrong. And even if you do consider these things, you shouldn't over-estimate your ability to predict all potential consequences of poor quality work.

Well designed, clear and clean code is probably one of the best protections from future unknowns. It's generally easier to understand, less likely to be hiding defects and easier to change. Continuously writing poor quality software and taking short-cuts is a recipe for chaos.

The reason these conversations are so frustrating is if someone is particularly entrenched in this stance, it's hard to influence them until something has gone wrong. By that point, it can be too late.

Fast feedback, avoid frameworks, re-evaluate regularly - lessons from a hackathon

Last weekend I gave up all claims to a social life, and took part in my very first hackathon.

Q. What's a hackathon?

A. It's an event where teams form to build rapid software prototypes that showcase a new idea or concept e.g. for a business, or for community development. The event takes place over a very short period, normally a couple of days.
This weekend the theme was building mobile applications using HTML5 and the Cloud. Ideas were presented on Friday evening, attendees formed teams, and teams built working prototypes to present on Sunday. There were a mixture of developers, designers and marketers taking part.

I chose to join a team porting a great iPhone app called Opuss to other mobile platforms .

The teams hard at workAn Opuss UI sketch

The weekend went well. I got out of it exactly what I'd hoped for in terms of new expertise, contacts, and inspiration.

Nevertheless, there were some hard lessons so I've summarised a few:

  1. Focus on feedback
    Some commercial software projects focus on delivering 'potentially shippable products' - things your user can use straight away - at the end of a cycle of work.

    Forget that.

    Focus on delivering a product that can generate feedback. For example even if your users will need login accounts for your application to be viable, don't build functioning login won't learn anything about your idea from that.

  2. Forget frameworks
    I'd done my research, the best approach would be to use the Sencha Touch framework to build our app, and to use PhoneGap to package it and deliver it to say Android.


    Learning Sencha Touch proved a big task, and it would have been much faster and simpler to write raw HTML, CSS and JavaScript. Use the simplest technologies you can, even if they aren't strategical for building a well-designed program that is maintainable over the long-term.

  3.  Re-evaluate regularly
    If it's been a long-time (a few hours) since you delivered a new version of the prototype and got some feedback, then stop. Think about why. Think about whether you can simplify what you're working on now, or even whether you really need to do it all. In our case, we could have abandoned Sencha Touch much earlier, and abandoned connecting to our servers API to logon and get a session ID etc.

All of these lessons probably aren't surprising to read. They could have come straight out of a Lean Startup text. It was a great reminder of principles I thought I had already taken to heart though.

XP Day for a first-timer

When I turned up at 'XP Day 2011', I was a little bit anxious. I didn't really know what it was, and didn't know anyone going. I had signed up out of curiosity, after seeing it mentioned on Twitter a few times.

Luckily, as it turned out, I hadn't inadvertently joined a cult or wandered into something else suitably tragic. I'd ended up in an 'open-space' format conference, with what looked like 'normal' people.

The idea was simple:

  • at the start of each day, attendees were able to propose topics for discussion, which were then allocated a room and timeslot
  • during the course of the day, attendees were free to roam between sessions that interested them (the law of two feet - if you're not learning or contributing, feel free to move on)
  • at the end of the day, everyone would head to the pub 

Dan Hill, of talking about Cohort Analysis

goldfish bowl conversation  on conceptual integrity

The sessions that transpired were a mixture of off-the-cuff conversations and pre-prepared presentations followed by discussion. Thanks to the law of two feet, I managed to get involved with everything from a coding dojo to a discussion on using cohort analysis to track customer engagement.

Unfortunately, I didn't manage to host a session myself, but the open-space format did, at least, mean I could get actively involved in other people's sessions.

I didn't feel out-of-my-depth or intimidated, in fact everyone was remarkably friendly. If you're a complete novice when it comes to agile or lean development or someone reluctant to speak up in unfamiliar situations, then I probably wouldn't recommend this event. Its quality is heavily reliant on a supply of willing and knowledgeable orators, and there was even some deliberate discouragement for rank amateurs: the registration form, for example, requested information on your personal experiences.

If you're not afraid to get involved though, definitely go along!

P.S. If you haven't heard of it, check out the eXtremeTuesdayClub, which meets every Tuesday. A number of the XP Day organisers, some of the most prominent members of the agile community, are regulars there.