Agile and the horizon effect
The 1960s saw the first ideological skirmish in computer chess programming (and by extension much of the nascent field of AI) between two schools of thought: ‘brute force’ and ‘selective search’. Brute force methods involved looking at every possible position on the board, whereas selective search advocated pruning the game tree by ignoring moves that looked plain wrong.
With the hindsight of Moore’s Law, this was never really a contest. Brute force’s superiority was reinforced with each new clock speed, and this is how all chess programs work today. Each move is considered, as is each reply, and so on. A computer will typically analyse ~250 million positions per second, evaluate them all and choose the computationally best branch of play.
Early brute force machines were set to calculate all variations to a fixed depth, such as five moves. However, programmers soon found that this seemingly fool-proof method was still leading to some terrible chess. The cause was a phenomenon dubbed the horizon effect, whereby the losing move lay beyond the point at which the computer stopped calculating. A computer playing to a fixed depth may therefore set out on what seems the best path, unaware of the disaster lurking around the corner. Frustratingly, it even may ‘see’ the losing move but find a way to delay it by a couple of meaningless forcing moves, thus pushing it beyond the fixed horizon. Out of sight, out of mind.
For humans, the horizon effect isn’t much of a problem. Intuition plays a surprisingly large role in chess, and experienced players can vocalise when a position “feels like trouble” even though the fireworks may be a few moves off. Famous studies by Adriaan de Groot show that much of this intuition is based on pattern recognition, so that over time a skilled player builds a pattern library and, with it, an innate early warning system.
Programmers, of course, wished to mimic this intuition in computers, and did so by introducing a concept known as quiescence; in effect, a measure of a position’s stability. At the end of each variation, quiescence is calculated. If the position is placid (quiescent), the variation can terminate and work starts on another branch. However, if the position is still deemed to have danger in it, the computer is allowed to look a little further, until it again finds a quiescent state. Quiescence fills the role of the human’s alarm bells, and substitutes for the intuition that certain scenarios are going to need a bit more care to solve.
Any system where work is conducted to a fixed depth is susceptible to the same effect, and of course Agile is no exception. As we all know, Agile often doesn’t afford us a long discovery phase, and asks us to focus on short, practical iterations. This goes against a designer’s natural instincts; one of the more common complaints designers have of Agile is that it rarely gives us the chance to conceive an over-arching ‘solution’ of the problem space. The horizon effect again. We work on each piece, not knowing whether or not disaster lies around the corner. We have to delay solving potentially tricky problems, and can never be really sure that the site will work until we’ve completed it.
Although in theory Agile is comfortable with the idea of rework, the real world penalties are high. At worst, we might have to scrap a whole approach because of an unforeseen problem in a future iteration, repeatedly pushed beyond our horizons until it is too late. Try telling your clients that the last £10,000 they paid you were wasted and see how far theory gets you.
As good designers, we therefore need our own quiescence search. Just as the computer develops an intuition for choppy waters ahead, so must we. We build up a box of tricks to handle these scenarios: starting work on tricky stories early (while keeping it secret from Agile dogmatists!), pushing easier user stories up the chart to buy us time, and so on. But these techniques only come with years of experience. As with the chess player, we rely on pattern recognition, experience and skill to act as our early warning system and flag difficult stories in advance.
The more I think about Agile design, the more I’m convinced it needs senior staff. Send a junior IA into the middle of an experienced Agile team and they’ll struggle to keep their heads above water. With senior design staff still at a premium, I suspect many companies will have to compromise the integrity of either their user-centred design or their Agile processes. I’ll leave it to you to decide which is more likely.
