Monthly Archives: April 2010

Not Time Boxing or Commitments but Managing Risk by Using SLAs

Recently, I’ve been working closely with one of the teams here and consequently have lead some changes to the dashboard that they keep for use in their planning sessions with stakeholders and for my benefit in being able to track costs associated to any client development we’re doing. Having been keeping the data in this team now for about a year it’s proved incredibly useful.

Owing to the nature of the stakeholders that provide direction in respect of the teams’ pipeline, we’ve often seen situations where new features have been prioritised over features that have been in the backlog for some time, something that is quite normal I’m sure. In turn though, this has then meant that when the team does come to start the work pertaining to the features that were entered in to the backlog first, the delivery date that was originally suggested to the external client will have already passed. In those cases, whilst the feature has been estimated using “T-Shirt” sizing we’ve only then tracked how long the team took to do the work. This is not a problem in itself, but a recent observation made by the team was that they felt that it didn’t necessarily help them make informed decisions in terms of how to deliver the feature.

A while ago, the team stopped using iterations opting for more of a flow based model, we’ve been implying a Work in Progress limit by restricting the amount of streams available to feature work but because of Hidden Work previously discussed, amongst other things, these often failed to focus the team in respect of delivery too.

In a recent prioritisation session with the stakeholders it was decided that we would focus efforts on improving the performance of a couple of the endpoints within an API.  It was decided that owing to the exploratory nature of the work, the developers undertaking the work would do a time boxed spiked.

It was at this point that I started seeing a possible way of getting the team to focus in on a delivery date. At the time, I thought that it would be reasonable to use the data relating to the estimates that we’d previously made to also time box future feature work. As can be seen in the image below, in the team’s dashboard we display the minimum, maximum, and standard deviation (variance) for all features within a T-Shirt size. Extending that  so that we used it to make a commitment to our clients / stakeholders that for example, anything deemed to be be a medium would be done in 19 days, the mean for that size.

Days Taken, Committed to Done by T-Shirt Size Including Variance

Thankfully, I realised the mistake I was making. I was suggesting that a team should take a piece of work which granted they had done some analysis on, and to make a commitment that they would deliver it in a set period of time. This is tantamount to asking a team to make a commitment at the beginning of an iteration to deliver a certain amount of functionality, based on whatever measure they are using at the time. To tend towards an iteration based delivery mentality is to tend towards one of two types of failure in my opinion: Under Commitment from the team so that they know they have more chance of meeting the expectations placed upon them; good for their moral but not great for the overall productivity. Alternatively, making a commitment which the team has no way of knowing that they can meet which can be demoralising for the team over time.

So What Instead Then?

The solution for us was born out of the fact that we keep so much data in relation to the features that we have delivered.

Recently, we have started to chart the data in a different manner to the one above, displaying the time taken for all features from when the team committed to doing them, to when they were released to the production environment (i.e. they were done).

All Features, Committed to Done

Visually, this is not necessarily an immediate improvement. The obvious thing that does now present itself though is just how much variance there is in the size of the features that the team is doing.

The chart is improved though when series displaying the Mean and Standard Deviation are added, see below:

All Features, Committed to Done with Mean and Std Deviations

However, it is still just a chart showing a lot of variation which when you consider all the features ever delivered by the team, you would expect and the Standard Deviation which needs no explanation. Some might be questioning what use the above is at all, well the benefit for us is in reinforcing the volatility and inherent risk in any estimate. This is complemented massively by the table below.

Table Showing Cycle Item by Type

Above, the information we use the most is Number of Items; the size of sample set, the Average Plus Standard Deviation; the average Cycle Time for an item with an allowance for some risk and the Percentage of Items Greater than the Average Plus Standard Deviation; the risk in using the number as the basis for any estimate.

When combined this data allows us to do one important thing for our stakeholders. In the absence of any deadlines we can say that we endeavour to deliver any item according to its type within a certain timeframe (arithmetic mean for the sample set + 1 standard deviation), hence the team are defining an SLA.

What’s important to note though here is that the SLA is not a commitment; it is information relevant to the team to enable them to focus on the risk that is always present in delivering software. These dates are being used currently by the the team in planning exercises as indicators and by putting the dates on the cards that go through the wall (using a post-it note on top of the card) to help them focus on delivery and furthermore and perhaps more importantly, to allow them to manage the risk on a daily basis. At the stand up when talking about cards they can focus specifically on how to increase their chances of delivering the feature within SLA.

It’s early days for this experiment and there isn’t enough data as yet to suggest that it is either succeeding or failing. Anecdotally though, it seems to be having a positive impact.