How applying Theory of Constraints helped us optimise our code

The neck of a bottle of prosecco in front of a fire.

My team have been working on improving the performance our API, and identified a database call as the cause of some problems.

The team suggested three ways to tackle this problem:

  • Scale up the database till it can meet our requirements.
  • Introduce some light-weight caching in the application to reduce load on the database.
  • Examine the query plan for this database call to find out whether the query can be optimised.

Which of these should we attempt first? There was some intense discussion about this, with arguments made in favour of each approach. What we needed was a simple framework for making decisions about how to improve our system.

This is where the Theory of Constraints (ToC) can help. Originally expounded as a paradigm for improving manufacturing systems, ToC is really useful in software engineering, both when managing projects and when improving the performance of the systems we create.

Theory of Constraints

The preliminary step in applying ToC is to identify the Goal of your system. In the case of this API, the Goal is to supply accurate data to consumers.

Now we understand the Goal of the system, we can define the Throughput of the system as the rate at which it can deliver units of that goal, in our case API responses. We can also define the Operating Expenses of the system (the cost of servers) and its Inventory (requests waiting for responses).

The next step is to identify the Constraint of the system. This is the element in the system that dictates the system’s Throughput. In a physical system, a useful heuristic is a build-up of Inventory in front of this element. In our API, our monitoring helped us pinpoint the bottleneck.

The next three steps give us a sequence of approaches for tackling the Constraint:

  • First, Exploit the Constraint by finding local changes you can make to improve its performance.
  • Second, Subordinate the rest of the system to the Constraint by finding ways to reduce pressure on it so it can perform more smoothly.
  • Third, Elevate the Constraint by increasing the resources available to it, committing to additional Operating Expenses if necessary.

Exploitation comes first because it’s quick, cheap and local. To Subordinate you need to consider the effects on the rest of the system, but there shouldn’t be significant costs involved. Elevating the Constraint may well cost a fair amount, so it comes last on the list.

Once you have applied these steps you will either find that the Constraint has moved elsewhere (you’ve ‘broken’ the original Constraint), or it has remained in place. In either case, you should repeat the steps as part of a culture of continuous improvement. Eventually you want to see the constraint move outside your system and become a matter of consumer demand.

Applying ToC to our question

If we look at the team’s three suggestions, we can see that each corresponds to one of these techniques:

  • Scaling up the database is Elevation: there’s a clear financial cost in using larger servers.
  • Introducing caching is Subordination: we’re changing the rest of the system to reduce pressure on the Constraint, and need to consider questions such as cache invalidation before we make this change.
  • Optimising the query is Exploitation: we’re making local changes to the Constraint to improve its performance.

Applying ToC tells us which of these approaches to consider first, namely optimising the query. We can look at caching if an optimised query is still not sufficient, and scaling should be a last resort.

In our case, query optimisation was sufficient. We managed to meet our performance target without introducing additional complexity to the system or incurring further cost.

Further Reading

Goldratt, Eliyahu M.; Jeff Cox. The Goal: A Process of Ongoing Improvement. Great Barrington, MA.: North River Press.

Reinvigorating a daily stand-up by walking the board

I’ve been working with a team who had a problem with focus: when I joined them, they seemed to be busy all the time, but they were frustrated that they weren’t making progress towards their sprint goals.

This situation could be seen in microcosm at the daily stand-up meeting. In this post I’m going to describe how a simple adjustment to this meeting helped us start to improve focus, morale and productivity.

The Scrum Guide gives a template for the daily stand-up meeting (which it calls the Daily Scrum):

The Daily Scrum is held at the same time and place each day to reduce complexity. During the meeting, the Development Team members explain:

  • What did I do yesterday that helped the Development Team meet the Sprint Goal?
  • What will I do today to help the Development Team meet the Sprint Goal?
  • Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?

My team was indeed practising this technique, but it seemed that they often forgot the discipline of focusing on ‘helping the Development Team meet the Sprint Goal’, and the meeting often descended into yesterday-I-diddery, with each team member recounting all the things they did the day before in trivial detail.

It seems to me that in a stand-up of this format, each team member’s incentive becomes having something to say, rather than showing progress towards the Sprint Goal, and this produces an incentive to be busy, no matter how irrelevant or frankly counterproductive the tasks might be. In this team, I saw a lot of effort spent on support tasks—whether or not the issue was pressing—, a significant amount of aimless ‘refactoring’, which was essentially yak shaving, and a tendency for team members to interrupt each other for help with lower priority work. In effect, everyone starts prioritising busy work, rather than focusing on the team’s goals.

The other consequence of this approach was that the team’s board was a poor representation of our work: people would be working on tasks that weren’t visible on the board, and the stories that were on the board often didn’t move anywhere. The Scrum Master and I tried various approaches to coordinate the board and the stand-up reports, but the focus was still lacking.

I’ve previously worked in a Kanban environment, and the format of a Kanban standup is significantly different:

Standup meetings have evolved differently with Kanban. The need to go around the room and ask the three questions is obviated by the card wall. … The focus is on flow of work. The facilitator … will “walk the board.” The convention has developed work backward—from right to left (in the direction of pull)—through the tickets on the board. The facilitator might solicit a status update on a ticket or simply ask if there is any additional information that is not on the board and may not be known to the team.

(Anderson, David J. Kanban, Successful evolutionary change for your technology business)

I suggested that we try this approach for a week, and see whether it helped give us more focus. As people were concerned that we might lose sight of important work, we agreed that we would walk the board first, and then run quickly round the team to see if anything was missing.

The initial results were encouraging, and several weeks later we are still walking the board, rather than going round the team. In particular, our board now contains a great deal more information on the tasks in play, and the team have got really good at carding up even small tasks so they are visible at the next stand-up. The amount of off-plan and busy work has also dropped, and this also be a result of the focus on the tasks on the board. Perhaps my favourite development is that the tasks on the board have become much smaller: the drive to get things done is now focused on pulling small, focused tasks across the board, rather than doing busy work.

Of course this technique is no cure-all, and it took a while for the team to acquire the discipline of walking the board in order, rather than jumping in to discuss whichever task particularly excites them. However, as an incremental adaptation, I’m very pleased with its results.

A Retrospective in the Park

The other day, I facilitated a sprint retrospective in the park. The sun was shining, and we had all been working hard to complete our backlog, so it felt like a nice reward for everyone’s efforts. Holding a retrospective outdoors can also give it an energy and sense of enthusiasm that is harder to find in a small room.

I’ve run outdoor retrospectives before, and have previously followed fairly classic plans, with much arranging of index cards. This has never been a great success, as the slightest breath of a breeze can make a mess of your planning. For this retrospective, I designed a plan to avoid these problems, drawing some ideas from the Appreciative Retrospective plan.

This retrospective took an hour for a team of nine. You’ll need a pile of index cards or sticky notes, and a pen per person. Here’s how to do it:

1. Choose your location

Some people are sun lovers, whilst others, like me, burn easily and need some shade, so find a location that will work for everyone. Don’t worry too much about the state of the grass, as I suggest you conduct the retrospective standing up, if possible.

Get everyone to stand in a circle, with enough personal space for everyone, but close enough that you can hear everyone speak.

Observations: It’s nice if you can find a location where there won’t be too many distractions. We weren’t entirely successful: there was a hen party in another corner of the park, whose popping of prosecco corks and parading of an anatomically exaggerated blow-up mannequin was hard to ignore; there was also a group of male models sunning themselves noisily behind us (I’m working at a fashion company, so this is less unusual than it may sound), and three young women were smoking some interesting cigarettes upwind of us. Nonetheless, our retrospective was a success despite occasional distractions.

2. Characterise the sprint

Ask everyone to spend a couple of minutes coming up with three words to characterise the sprint; then go clockwise round the circle and ask each person to tell the team their three words.

Observations: It’s surprising how much difficulty people have sticking to three words; the important focus of this task is not the three-word limit, but getting a concise summary of the sprint.

3. Thank your neighbour

Moving anticlockwise around the team, ask each team member to thank their neighbour for something they have done during this sprint.

Observations: Our Scrum Master broke the rules by thanking the whole team for their efforts. The focus of this task is to generate a positive mood across the team, and it’s important that no one misses out on individual thanks, so I asked him to thank his neighbour for something as well.

4. Describe what went well

Hand each person three cards, and give them three minutes to write down three things that went well during the sprint. Going clockwise round the circle from a different starting point, ask each person to read out their three successes.

Observations: It’s useful for the facilitator to observe and comment on common themes, as this can help reinforce good practice.

5. Describe what could improve

Hand each person three more cards, and give them another three minutes to write down three things that could have gone even better during the sprint. Then go anticlockwise round the circle and ask each person to read out their three improvements.

6. Group the improvements

Instead of arranging the cards on a whiteboard (which isn’t practical in the park), appoint a champion for each improvement. Ask the first person to choose one of the improvements they suggested, and then get everyone else to hand this person any cards that describe a similar improvement. Keep running round the team until each team member has just one group of cards.

Observations: There’s a chance that you’ll end up with more themes than team members, in which case you’ll have to make a decision to drop some of these themes; in our case we had fewer common themes than team members, so we didn’t have to do this.

7. Select and discuss the most common themes

Rather than dot-voting, which again is impractical in the park, select the commonest themes. Ask everyone with any cards to take a step into the circle. Then ask everyone with just one card to take a step out again; then everyone with just two cards; then three, and so on until just three people are left in the inner circle.

Then have a three-minute discussion of each of these suggested improvements, with the focus of identifying at least one action per theme for the next sprint. Ensure someone is assigned to each action.

Observations: We ran slightly over the three minutes assigned to each theme, but this wasn’t a problem; if we hadn’t had a time limit, I suspect the conversation would have been much less focused.

8. Round off the retrospective

Finally, going round the circle clockwise, ask everyone to describe how they felt about the retrospective itself.

Observations: The feedback was very positive. The team had clearly enjoyed the opportunity to get out of the office, and they felt that the session had been successful: everyone was engaged and we came up with some good actions.