Do we need clearer roles and responsibilities?

Why are people are still preaching that better, more clearly articulated roles and responsibilities will improve productivity? Provide a transparent environment where people have a clear understanding of what they’re working toward, and its value, and you’ll get better results.

As I write this, I’m sitting on a train travelling home. I’ve just been through the blog posts in my RSS feed. Three offer very similar advice about how to improve an individual’s productivity in the workplace. Their answer to this perennial conundrum is that we need to provide clearer objectives and roles and responsibilities on a daily basis. Given that the posts originate from people promoting Agile as a more enlightened way of working, I shake my head in wonder. Really, I do.

The value of roles

There are times when clear roles and responsibilities are essential – this post is not to suggest that their use be completely dropped. Consider the outcome of the incident now known as the Miracle On The Hudson; having just taken off from New York City’s LaGuardia Airport, a US Airways aeroplane, flight 1549, struck a flock of geese. Both engines failed. Some 3 minutes later, the Captain and his crew, a First Officer and 3 Flight Attendants, had performed an emergency landing on the Hudson River. In an air crash, speed, accuracy of trained response and calm are paramount. Although the First Officer had been flying at the time, the Captain took the controls, leaving the officer to go through a 3-page emergency procedure to restart the engines. The three flight attendants showed passengers how to brace themselves for impact and took control of evacuating the plane, even as it filled with water. The captain walked up and down the sinking aircraft to be sure no-one had been left behind and was the last person to leave. Each of the crew members was helped by their training and their knowledge of their specific, unique roles and responsibilities.

The difference for IT

The problem, is that few business teams – IT or otherwise – face the same kind of challenge as existed for the air crew. In IT, problems tends to require a creative response, one that cannot be considered in advance, which no training will prepare us for, but which must be invented anew each time. Strict roles and responsibilities don’t help with this. Indeed, they may hinder it because instead of thinking freely, people are prescribed. Control does not sit well with creativity, originality or motivation. Attempting to control an individual in a group setting reinforces a silo-based mentality to work. That means that the attempt to control an individual through a strictly defined role and associated responsibilities, affects the whole system. In my opinion it compels people to work within the boundaries that are set for them – intentionally or otherwise.

Do rigid role boundaries assist in speed?

In the air crash example, knowing who is responsible for each task helped speed. Proponents of clear roles and responsibilities suggest that decisions are made faster and are accepted with less confusion. Yet in my experience one of the negative results is a reduction in collaboration. The classic case of this, which I am sure we have all experienced, is somebody justifying their inaction with: “well, it’s not my responsibility”. I’ve known many cases where people fought NOT to take responsibility for an unpleasant job or fought over who TOOK responsibility for a prestigious job and the result was always that the product slowed or sometimes stalled as collaboration evaporated.

Individuals can always change the rules

So far I’ve focused on the work. But software development is ultimately a people-orientated endeavour. I was thinking about this as I sat with my son – a four-and-a-half year old, to do his numeracy homework. For some reason he was writing a lot of his numbers back to front. I kept on trying to correct him and at one point said to him “just write me five nines”. His response was to write the number 5 and 9. Perfectly, I might add. Some might say it showed intelligence. I’m inclined to say that he was being subversive. Either way, it was a proud Dad moment. But that’s one of the problems with the advice being offered: people don’t like being told what to do. They either do what they think is best at the time, or in a small amount of cases intentionally subvert the system.

Demotivating boundaries

When we’re children recognition from our dads and teachers motivates us, as we grow older the influence of our peers is what counts. I tend to perform best when I know I am there to make a defined contribution. To this end, a degree of definition of my role and responsibilities helps, but a rigid boundary is demotivating, because it constrains my ability to contribute as much as I could or would like to. This demotivation is exacerbated when people have to discover where their responsibilities lie. I’ve heard people refer to this as being knocked back – they’ve tried to do something which is perhaps of interest to them, or where they perceive there to be an opportunity, only to be told that their contribution wasn’t wanted.

Doing things differently

So what instead? As with all pernicious problems the first step is to observe what is happening. You might find people in a stand up talking about the piece of work they’re doing, rather than the work the team needs to do. Perhaps you see executives or departments becoming more rigid in role demarcation. A lack of vision or sense of purpose might pervade the organisation. Worst of all, perhaps no-one thinks making things better is any part of their ‘responsibility’. Cause and effect is difficult to establish in a software development scenario, but you should be able to state your observations and assumptions. Establish what effect an improvement might have. Test your assumptions with others – what’s their opinion? Don’t fall in to the trap that it is your role to change people. Remember, the way in which people behave is far more likely to be driven by the environment around them than anything else. Once there is agreement in the group about the perceived problem, consider what you might change to bring about improvements. Test whether these improvements have had the desired effect. Respond to the feedback you receive. Rinse and repeat.

I know that those advocating clarity of roles and responsibilities see the same problems in demotivation and wasted efforts as I do. The difference is that they believe more control is the answer, while I believe that less direct control frees people to create their own solutions. Those 5 crew members on US Airways flight 1549 worked as a team, each within their specialisms, and brought off “the most successful ditching in aviation history”[1]. Software development teams need to be more cross-functional in nature, but I believe that they too can perform their own miracles – as long as they have an environment that supports them rather than dividing them.

Thanks go to the inimitable Joshua James Arnold, Lady H of Edmonton (but of no fixed Twitter abode), and the sartorially precise Mark Krishan (50 shades of) Gray for helping me to improve this blog post.

[1] New York Post, 2009. Quiet Air Hero Is Captain America. [online] Available at: <http://www.nypost.com/p/news/regional/item_Goem4fAiUd2hsctASfAjGJ>. [Accessed 28 November 2012].

Using The Voice Of the Customer As Input For A Retrospective

Previously, I have worked with a team whose overall effectiveness has been called in to question by people who the team interfaced with. Upon further investigation it became clear that the perception those people had of the team was largely based on anecdote and yet, as the lead of the team said, perception is reality.

The lead and I sat to discuss this and agreed to substantiate some of the comments being made and the process that we would follow to derive actions that we could then take towards discernible improvements. I think that what we ended with is an interesting way of looking at gaining input in to a team’s retrospective and contrary to the way I have seen it done previously, namely that the team sourced opinion from outside of the team rather than internally, and so wanted to share it.

We started by identifying the customers of the team and given that this was the first instance of running this process, we also chose to keep the group size small, identifying just the key individuals with an agreement that in subsequent iterations, we would extend the group.

We designed a small survey comprising 4 questions that we would ask of the individuals:

1. What is your perception of the team’s performance?
2. On a scale of 1 to 10, how would you rate the performance of the team?
3. Given the rating that you have just given the team, how could we make ourselves a 10?
4. Can you suggest any ways in which we could measure the effect of the improvements that you suggest?

Questions 1 and 2 in the above were actually of little significance in respect of helping the team to improve, their purpose focused more on providing some context to the presentation that would be made back to the team after the interviews had been conducted.

Question 3 gave the person being interviewed an opportunity to make specific recommendations for how the team could improve and so in our opinion, was the most important. Less important then was question 4 though it did serve to make the people being interviewed question the feedback that they had in response to question 3 and to offer the team with some sense of measures that they could use that would be meaningful to their customers.

With this data, the team lead conducted a Retrospective. It started with a presentation of the information that had been collected. The team was told from the outset that this was data that had been collected and that while they may disagree with some of the comments made, it was the perception of others and that therefore they needed to accept it as it was.

The remainder of the Retrospective was split in to three sections, an idea generation session titled “What Changes Can We Make That Will Result In Improvement?”, followed by an associated filtering session which looked at the ideas generated and asked the question “How Will We Know That The Change Is An Improvement?”. The purpose of this filtering session was to filter out those ideas that could not be measured one way or another. The team estimated the value an item would have in respect of impacting their performance using a relative scale and followed that by estimating the effort involved in bringing about the change. Both of these estimates were done using a relative points scale. Finally, the value estimate was then divided by the effort estimate to give an indication of the Return on Investment and the item with the highest return was selected for action by the team.

Experimentation: A Pattern for Affecting Measured Change

Note: This post is based almost entirely on one of the things that I took away having attended a talk titled “Agile Coaching Tips” by Rachel Davies held at Skills Matter but perhaps overlooked at the time. As such, the originating ideas are not mine and instead, Rachel’s. This post is more something that I have constructed as a result of my own recent thinking, primarily driven by my interest in the kanban community and talking with people like Benjamin Mitchell which has led to me growing more appreciative of the value of qualitative and quantitative measures. This in turn has led me to revisit one of the subjects that Rachel introduced in the talk she gave, “Making Change As An Experiment” (See between 38:36 and 42:09 in the video I have linked to.)

Why Experiment?

Rachel proposed, and I agree with her, that talking about change with somebody can often result in them becoming wary whereas using nomenclature, in this case talking of experimenting with something different, instead of using the word change can make those people involved and or potentially affected feel more comfortable. In that respect, I like the idea of using experimentation as a means to effect change within what I do as primarily, it is considering the feelings of the people that are involved in that change up front. So often in my experience that not insignificant factor can be overlooked.

Secondly, when thinking about experiments that are run in a laboratory, it is accepted that you may fail to meet your objective; you’re questioning whether you can change some thing’s existing state. In relation to that. because you had a hypothesis that were looking to prove or disprove, you will have had to measure the outcome of any actions you performed and the conditions they were attempted under.

With those 2 points in mind, I’ve taken Rachel’s original idea and noted how I think it could be used as a pattern to bring about measured improvements.

Experimentation As A Pattern

1. Understand the Current State

The first step in my opinion, is to understand what it is you are attempting to achieve overall (your vision). From there, the next step is to understand the first step (perhaps the first impediment you want to remove or the first improvement you want to make otherwise) you want to take you towards your vision. Techniques such as Retrospectives, The 5 Whys and A3 Thinking amongst others, can help here but above all, the one tool that I keep coming back to currently is Force Field Analysis.

2. Hypothesise

The received wisdom is that hypothesising should be the process that is used to understand the goals of the experiment with which I would agree. Important to me here though is that the goal should be something that can be clearly articulated to anyone, i.e. in the case of a development team, those outside of the team from a non technical background too. It is possibly worth noting the goals too, this is not something that I have personally done but I do see that it might have brought value to previous change efforts.

It is particularly key here that if the experiment you wish to run is related to a practice within a team, everyone within the team understands why you are wanting to introduce a new practice or change an existing one. Furthermore, it is important in my opinion that everyone in the team has an opportunity to discuss how you might change. This is important so that everyone is in a position to question the validity of their own and that of their peers approach on a daily basis throughout the experiment. In my opinion, it is for all those involved in the experiment to ensure that it runs as defined.

3. Plan The Experiment / Decide On How To Measure Effect Of Change

The idea that any change in practices or process should be measured is, as I describe above, one of the most important aspects of why I think that experimentation is a useful way of looking to introduce change. Based on the findings from Understanding the Current State and the goals derived from the work done whilst Hypothesising you should be able to define either quantitative or qualitative set of measures. It would be ideal in most cases if you were able to identify measures that you already have in place so that you can compare and contrast, but not essential. In the absence of existing measures, I would suggest that it is worth extending the duration of the experiment so that you can observe change over time.

By way of a simple example, let’s imagine that you or the team you work within notice that features are batching when they get to the tester and perhaps even that in a lot of circumstances, the features end up having to be reworked. The overall affect of this is that the team is unable to concentrate on other work and that the features themselves take longer to be delivered unnecessarily. Some measures you might consider are: The amount of re-work, for example the number of times that a feature is passed between a developer and a tester; the number of bugs raised against a feature whilst it is being developed and finally the Cycle Time for features.

Finally, decide how long the experiment will run for. As I mention above, be mindful of how long you will need to gather data that will be statistically significant. If you already have a good set of data to compare any new metrics with then you can (though not necessarily should) use a shorter period of time and conversely, I would suggest that in the absence of any data to start with you should consider leaving the experiment to run for as long as possible.

4. Run the Experiment

As I’ve suggested in the sections above, I think the most important things to be concerned with whilst running the experiment are as follows: Have a clear vision for what you’re trying to achieve and be able to articulate a set of discreet measurable goals for the experiment. Ensure that, on a best endeavours basis (acknowledging that some people will always be difficult to be persuaded of the benefits of one practice over another. See Diffusion of Innovation, the Five Stages of the Adoption Process) everybody in the team is signed up to the experiment and know that they have a role to play within the experiment; time box the experiment so that those that aren’t necessarily persuaded of its virtues know that it is for a set time and furthermore to ensure that the amount of energy expended on an invalid hypothesis is minimised; take time at regular intervals to ensure that you remain true to the practices or process improvements that you are trying to bring about, without doing this it will invalidate the experiment.

Finally, at the end of the experiment take another opportunity to review what happened throughout the experiment, not just from the perspective of the measures that you have in place but from the perspective of those involved too; how did everyone involved feel the experiment went and what benefits did they feel it brought? In addition, my advice would be to be as transparent with all affected parties as possible with the measures that you are using and the results that you publish. I think this is important if for no other reason than it might generate further discussion, particularly outside of the team.

5. Iterate

It’s important to recognise that what is learnt from the experiment is the most important thing, as opposed to whether or not you succeed in meeting your goals. If you successfully met your hypothesis that’s great, but if you didn’t, ask yourself whether there is still value in what you are trying to achieve? If there is, can you change your previously defined practices to derive the value you are hoping to? Importantly too, are the measures that you defined still valid, are there ones that could be replaced with others, alternatively are there any that can be added to allow you to better understand performance against your goals?

In Conclusion

I am sure that the using experimentation as a pattern for change is something that others including Rachel have suggested before and so what I am proposing is nothing new, it is also without doubt that making change using the technique that I suggest adds overhead to the change process itself. I do firmly believe though that using experimentation as a technique to introduce change is beneficial owing to its focus on using measurement which allows those involved to be objective in their appraisal of any impact and that lastly, it considers the people involved up front thinking about the fact that when confronted with it, most people are wary of change.

Output from the Session I Proposed at the UK Agile Coaches Gathering, 2010

At the recent UK Agile Coaches Gathering (held at the magnificent Bletchley Park), the theme for which was Helping People Grow,  I proposed a session titled “Leaving a Legacy, How Do You Leave an Environment in Which a Team Can Continue to Grow” for one of the slots during the Open Space. There follows a summary of the session and then the notes that I captured, augmented from memory where possible.

To start I thought that I should provide an explanation as to why I proposed the session. In recent months, I have heard an awful lot of stories about change efforts that have been fantastically successful but when one thing is changed, perhaps a person leaving an organisation or the structure of the team changed, the momentum from that effort goes too and in some circumstances then leads to those involved moving backwards to their previous state. Furthermore and whilst I have never held the title of Agile Coach, from personal experience there seems to be a ceiling within organisations which when reached by a team, change becomes more difficult; typically because they are needing to challenge hierarchies or the received wisdom within that organisation.

Prior to the session, I did some preparation and thought about how as a coach you ensure you leave a legacy I came up with the following:

As a coach, you need to help a team learn how to change but that at the same time, you need to help an organisation learn how to allow that team to change

I then specifically listed a couple of points for each of Team and Organisation that I thought would ensure a legacy of a culture of continual change. From a Team perspective, I noted that I thought primarily they need to be taught how to be introspective and secondly and linked to the the first point, you need to also provide them with a tool set to facilitate their continued change. Form an Organisation’s perspective, I noted that you would need to help them understand their purpose, to help them understand the value they want from that team and lastly, how to communicate both those things to the team.

Once I’d set the scene with the above, we moved in to the discussion the notes of which follow:

• The team need to get to a point where they have the courage to stand up for what they believe in
• A coach should not become too embedded within a team
  • Doing this reduces the team’s reliance on the coach
• Internal learning within a team leads to a team encountering blockers outside the team
• Teams can be too busy building stuff to have a vested interest in removing the impediments outside their team
  • The above is most apparent in Scrum when the Scrum Master is delivery focussed (e.g. a Project Manager or Lead Developer) as opposed to being focussed on Process Improvement
• It is key that the team needs to feel as though they own the process
  • Process Smell: 1 person owning the process
• A lot of the examples given by teh attendees centred around organisations that were hostile to any change in the first place
• Even if you have somebody that really cares about improving the team, people, including those outside the team need to experience the benefit before they will actually assist in further improvement
• (Team reaching a ceiling) Awful lot of organisations that can’t articulate their vision
  • Means that they struggle to say no to additional work load as a consequence meaning that the org is just busy, no slack
  • No tools to resolve conflict
• It was noted that is is harder to help an organisation be able to define and communicate their values when they have existed for a long time
• Is the answer, in terms of leaving a legacy that you leave a team that have a clear set of values that they believe in?
• To test whether in a coach’s absence the team is still being successful the following might be considered to be good indicators:
  • Are they still working as a team
  • Are they changing things
  • Are they still focussing on quality, internal and external
  • Are they learning
  • Are they innovating
  • Can it still be said that they have confidence
  • It was noted that the tests above are valid at an organisational level too
    • e.g. Kanban boards spreading outside of IT departments
• Differentiating factor? If teams or the organisation is stressed?
• See image below, showing how trust of a team is derived by others, thanks to Rachel Davies
  • Reciprocal though is that the others need to know that you as a coach know what you’re doing
• We should accept that change efforts are incredibly fragile. e.g. When a new leader enters an organisation they will always want to make their own change
  • Credibility comes from evidence: gut feel is not enough
• Is transparency part of the answer?
• Are the goals of the team and the organisation aligned? If not, changes in practices will never flourish
• Is the thing that is missing conversation? In a hierarchical structure is the ability to hold conversations across all levels there?

Image: Trust Equation

Thanks to all the participants who came along to the session and made it what it was, I enjoyed it and learnt something too, hopefully the same can be said for yourselves.

 

Lessons Learned Working in a Service Company Servicing Large Projects

I was fortunate enough recently to attend LeanCamp, an unconference organised by Salim Virani and aimed at people interested in the Lean Startup meme. I was reminded the other day of one of the sessions I attended. “Lessons Learned Bootstrapping a Service Company” was run by Chris Parsons about his experiences in starting his company, Eden Development.

Of the conferences that I have been to recently, I’ve found myself gravitating towards sessions that take an experience report format more. I like them as they’re nothing more than observational. Whilst they do often focus on one subject in particular the people presenting are giving you an opportunity to listen to their experiences, good and bad of trying to do something one way or another and not just trying to sell you a mentality / ideology. I do find sometimes that they let themselves down in the respect that evidence provided of the effects is anecdotal instead of quantitative. Happily, this was a less formal session and so that wasn’t a concern.

At 7digital we’re starting to near the end of the first phase of a project that is large in comparison to the normal size of those that we do. To be specific, we’ve been involved in the project since June last year and have been actively working to deliver features for the client since early November. It has consumed at a minimum, 4 developers full time and for a couple of months, 8. As I suggest, not large necessarily in comparison to other projects within the industry, but large for us.

I was reminded of Chris’ session as I start to reflect on what I have learned in delivering this project. Chris outlined 5 key lessons in a slide deck and then spoke about them. I thought that I might use those headings as the basis for my own blog post in respect of what I have learned.

1. Learn How to Say No

Chris’ point in the presentation was that he wished that he’d learnt to say no more often to pieces of work that he took on in the formative stage of Eden.

I’m certainly not in a position to be able to make those decisions outright at 7digital nor for that matter would we as a company want to; we’re in a privileged position that allows us to work with some great clients on interesting pieces of work that suit all concerned.

In the case of this project in particular owing to the client’s incumbent delivery process, we’ve had to accept that we have to work in a way that is contrary to the way in which we’re happiest delivering software. In contrast to Chris’ point, I think what I’ve learnt or at least had reiterated to me, is that saying yes to something but in doing so asking why, is very powerful. On a number of occasions I’ve been able to ask why a practice is used and in doing so understand 2 things; the context that surrounded the practice and secondly, the way in which the practice is viewed. With those bits of information I’ve been able to bring some subtle but nonetheless tangible change to the project and hopefully have contributed to making it more successful as a result.

2. Hire Passion over Experience

I don’t have notes that have the specifics of Chris’ points here but remember agreeing wholeheartedly with him. I think he made the point that there was a correlation between those that were passionate about development and the breadth of their skill set and ability to pick up say, a new language.

There isn’t really anything within this point that I can speak about that isn’t only tenuously linked and so I am leaving comment in this section.

3. Spend Where the Value Is

On this point, Chris spoke about the fact that at Eden, rather than having invested in tailored office space from the outset as so many other startups do (along with presumably, micro scooters / space hoppers / foosball tables etc), they invested in things that mattered such as decent chairs and machines for their staff.

In deference to our typical approach to development, on this project we did a lot of up front analysis, some 2 or 3 months, which at the time felt a little uncomfortable. Reflecting upon this now I think we did about the right amount and instead of just documenting requirements we spent a lot of time challenging them in order to understand the problem that was attempting to be solved. We asked the client to give us ways in which we could measure success of the software we were to deliver and most importantly, we tried to understand what the smallest piece of work we could do for the client to suffice their requirement. This is not to say that we’ve done half a job, far from it. I think that whilst we were constrained to an extent by the project bias this piece of work took, we have attempted to embed an incremental approach where possible. I think that this effort was worth the investment of our time and money, we haven’t to date included the work done in any bill but it has paid for itself in my opinion. We ended up with a very discreet set of functionality that services a requirement and nothing more. We didn’t end up developing software that isn’t going to be used and the software that we have delivered is in the most part, measurable aiding our desire internally to automate all of our acceptance tests.

4. Cashflow

Here, Chris spoke about how having some savings as a company has helped Eden move from a position where they were chasing all types of work even that that was underpaid to one where they can afford to be a little more selective about the type of work or the clients that they work with.

For us in respect of this project, it’s certainly not work that we would have turned away. It has been a great piece of work to be involved in. My main lesson here has been that there is a hidden cost often not consider associated to development of this size in a department the size of ours. As I stated at the beginning of this article, this project has utilised up to 8 developers for a time which equates to roughly a third of the department. Because of the way the department is structured, we have 4 product teams and the fact that this work spanned 2 of those, it heightened the need for effective communication and the need for collaboration between the teams which has a cost associated with it. When considering work of this relative scale again, I will be sure to consider that fact.

There is also the matter that undertaking a large body of work such as this for us could have presented problems in our ability to undertake other smaller pieces of work; we might have missed out on opportunities elsewhere. Thankfully we haven’t, or at least we haven’t noticed any that we have.

5. Have a Higher Purpose

Finally, Chris spoke about the fact that in setting up Eden his desire was always to be building great software for clients. He went on to say that building a company is made up of a thousand small decisions and that it’s important to let the people who work for you to make those decisions though the only way that that would be successful is to have a clear vision that everybody knows and understands. He suggested that he has been known to “pop quiz” people within Eden on it.

In the Development Team here at 7digital we’re continually striving to produce software that is the right thing and is to a high standard of quality. From the moment that we set out on this endeavour we have strived aided by the Green Field nature of this work, for sure, to deliver against those aspirations and based on the measures that we have in place internally, I think we succeeded. One example that I would cite is this; we found that there was a misunderstanding owing to assumptions within everybody’s understanding of our acceptance criteria for one of the features. Unfortunately this wasn’t caught until fairly late on. Given that we had a suite of unit, integration and system level tests we were able to quickly understand the impact of this new requirement and go on to make the change. Having worked on many different projects throughout my career I’ve rarely before been afforded such confidence that a requested change would be successful and have so little impact. Anecdotal evidence I know, but enough for me in this case.

Not Time Boxing or Commitments but Managing Risk by Using SLAs

Recently, I’ve been working closely with one of the teams here and consequently have lead some changes to the dashboard that they keep for use in their planning sessions with stakeholders and for my benefit in being able to track costs associated to any client development we’re doing. Having been keeping the data in this team now for about a year it’s proved incredibly useful.

Owing to the nature of the stakeholders that provide direction in respect of the teams’ pipeline, we’ve often seen situations where new features have been prioritised over features that have been in the backlog for some time, something that is quite normal I’m sure. In turn though, this has then meant that when the team does come to start the work pertaining to the features that were entered in to the backlog first, the delivery date that was originally suggested to the external client will have already passed. In those cases, whilst the feature has been estimated using “T-Shirt” sizing we’ve only then tracked how long the team took to do the work. This is not a problem in itself, but a recent observation made by the team was that they felt that it didn’t necessarily help them make informed decisions in terms of how to deliver the feature.

A while ago, the team stopped using iterations opting for more of a flow based model, we’ve been implying a Work in Progress limit by restricting the amount of streams available to feature work but because of Hidden Work previously discussed, amongst other things, these often failed to focus the team in respect of delivery too.

In a recent prioritisation session with the stakeholders it was decided that we would focus efforts on improving the performance of a couple of the endpoints within an API.  It was decided that owing to the exploratory nature of the work, the developers undertaking the work would do a time boxed spiked.

It was at this point that I started seeing a possible way of getting the team to focus in on a delivery date. At the time, I thought that it would be reasonable to use the data relating to the estimates that we’d previously made to also time box future feature work. As can be seen in the image below, in the team’s dashboard we display the minimum, maximum, and standard deviation (variance) for all features within a T-Shirt size. Extending that  so that we used it to make a commitment to our clients / stakeholders that for example, anything deemed to be be a medium would be done in 19 days, the mean for that size.

Thankfully, I realised the mistake I was making. I was suggesting that a team should take a piece of work which granted they had done some analysis on, and to make a commitment that they would deliver it in a set period of time. This is tantamount to asking a team to make a commitment at the beginning of an iteration to deliver a certain amount of functionality, based on whatever measure they are using at the time. To tend towards an iteration based delivery mentality is to tend towards one of two types of failure in my opinion: Under Commitment from the team so that they know they have more chance of meeting the expectations placed upon them; good for their moral but not great for the overall productivity. Alternatively, making a commitment which the team has no way of knowing that they can meet which can be demoralising for the team over time.

So What Instead Then?

The solution for us was born out of the fact that we keep so much data in relation to the features that we have delivered.

Recently, we have started to chart the data in a different manner to the one above, displaying the time taken for all features from when the team committed to doing them, to when they were released to the production environment (i.e. they were done).

Visually, this is not necessarily an immediate improvement. The obvious thing that does now present itself though is just how much variance there is in the size of the features that the team is doing.

The chart is improved though when series displaying the Mean and Standard Deviation are added, see below:

However, it is still just a chart showing a lot of variation which when you consider all the features ever delivered by the team, you would expect and the Standard Deviation which needs no explanation. Some might be questioning what use the above is at all, well the benefit for us is in reinforcing the volatility and inherent risk in any estimate. This is complemented massively by the table below.

Above, the information we use the most is Number of Items; the size of sample set, the Average Plus Standard Deviation; the average Cycle Time for an item with an allowance for some risk and the Percentage of Items Greater than the Average Plus Standard Deviation; the risk in using the number as the basis for any estimate.

When combined this data allows us to do one important thing for our stakeholders. In the absence of any deadlines we can say that we endeavour to deliver any item according to its type within a certain timeframe (arithmetic mean for the sample set + 1 standard deviation), hence the team are defining an SLA.

What’s important to note though here is that the SLA is not a commitment; it is information relevant to the team to enable them to focus on the risk that is always present in delivering software. These dates are being used currently by the the team in planning exercises as indicators and by putting the dates on the cards that go through the wall (using a post-it note on top of the card) to help them focus on delivery and furthermore and perhaps more importantly, to allow them to manage the risk on a daily basis. At the stand up when talking about cards they can focus specifically on how to increase their chances of delivering the feature within SLA.

It’s early days for this experiment and there isn’t enough data as yet to suggest that it is either succeeding or failing. Anecdotally though, it seems to be having a positive impact.

The Cost of Hidden Work

I’ve been working with one of the teams here recently all of whom have been feeling that the pace at which they deliver features is slower than they would like. I’ve run a number of sessions in order to gather data about the work they’ve been doing and how it flows through their wall. During one of the sessions, we focussed on the types of work that they are asked to do of which support was noted as one flavour.

The team currently structure their board around the ongoing need to service this support need, see an image of their current implementation left. One of the 5 developers within the team will work solely within the Support Stream seen at the bottom of their wall for an undetermined amount of time.

The support items are typically raised through our issue management application and usually carry an expectation with them that they will be worked on immediately irrespective of their size. The team has tried implementing an SLA around response times in respect of support items before but for one reason or another (discussion of which is outside the scope of this article), it has never really taken hold (We have in the sessions discussed Classes of Service details of which will be covered in a subsequent post.).

Back to the team’s opinion that they weren’t delivering as fast as they might like though; in our discussions I stated that I felt that part of the reason was that in undertaking the support items they were undertaking a lot of what was effectively hidden work. It was work that was also implicitly being prioritised above all the other items that had previously been agreed as highest priority in a prioritisation session.

We didn’t manage to reach a conclusion on what to do about the support stream though and so the conversation continued outside of the meeting. One of the Developers in the team noted what he saw to be the pros and cons of the support stream which with his permission, I’ve noted below:

Pros

• less distraction to the rest of the team
• less confusion who’s looking into what
• faster responses
• fixed amount of resources dedicated to support – support cannot slow up feature streams
• 1 person dealing with each support request = less overhead in understanding the problem

Cons

• support rota not evenly distributed amongst team members
• faster responses = less pressure on business to ask for automated solutions
• pre-sale enquiries don’t fit well, so would still cause distraction
• 1 person dealing with each support request = issues are hidden from rest of the team

I then followed up with a rather lengthy response which is the reason this post has emerged:

My issue with support is this; support items are essentially features that are being requested. In that respect, they should be prioritised among all the other features. The fact that there is a specific stream for support suggests that they are automatically accepted as being higher priority than the other features in progress and that is certainly not the case. Furthermore in prioritising them above the other features those other features are affected.

I think that it is far more prudent to prioritise the work you are doing on a daily basis, as a team and include the support items in that prioritisation. In my opinion, it is not so important that the rest of the team are not distracted as per the first point in the pros section above and that instead what is important is that you are as a team working on the highest priority items. Only then will you ever work together as a team to establish a flow. In that, I recognise that there is absolutely value in disrupting the flow of another item to get a major bug fixed, as there is also value in understanding why you are dealing with a support request and in the case of the why, you will only ever understand that by having a discussion with the other members of the team. I would hope that once you understand why as a team, you can then incorporate any learning in to an improvement cycle on a far more regular basis than waiting for your next retrospective at which time the item may have either been forgotten in the larger context or not voted as the highest priority thing to be attended to.

To the points above though; I’ve already touched on the first point in the pros section and as for the second point, I think that a clear definition of the role of the person acting floating manner will clear this up. On the third and forth points in respect of faster responses; as I’ve touched upon above, this may well be the case in respect of the items that you now classify as support but the net effects are that the other items are slower to deliver, I don’t have any data to substantiate this claim but I will point you at Little’s Law which suggests that the number of items in process is closely related to the overall time taken to process those items. On the fifth point, this is in direct conflict with our ambition to deal with one of the biggest problems we face as a department, that a select few that know more about our software and systems than anyone else; as well as the received opinion that two heads are better than one.

In terms of the cons listed above, the first point I may not understand fully but if I do understand it, surely a weekly system of having a floating role that fulfils a number of duties (which we can define as a group but for example: Monitoring the support queue, leading the stand up in the morning in discussing the blockers, the support requests that have come in and discussion about the work that is in progress.) would give a fair split of the responsibilities. I think I agree with the second point made but I would add this, the “business” (which is a term we should not allow to propagate lest we fall deeper in to a them and us mentality) classifying a support item as a feature and having a conversation about its priority and how to achieve it as a team which is what I am proposing, would I hope lead to other solutions being suggested and indeed the emergence of those automated systems bit by bit. If as a team you recognise that you have a little longer to undertake the item, schedule among the other things that you are doing, deliver it in a timely manner but put the building blocks in place for something that you can extend as time goes by. I’m in agreement with the fourth point about pre-sales enquiries as are we all I think, as we’ve now included it in the activities that we map on the board. Pre-Sales is a first class activity that everybody should have visibility of and also, should be able to undertake.

How Has This Concluded?

Firstly, the support stream hasn’t been removed from the team’s wall (as yet). They do now a rota in place though that clearly shows who is acting in that capacity. Furthermore and most importantly in my opinion, they now do prioritise the work within the context of the other features and don’t necessarily respond immediately but at a time that is convenient. They’ve also stopped tracking support items specifically opting instead to consider them features which are sized thus give us more reliable data about lead and cycle times and again and importantly actual visibility of the capacity the team has to do work.

Plan The Work But Don’t Work The Plan

I tweeted yesterday something that I had overheard to which a friend of mine, Toby then responded. As Toby then went on to suggest, the five tweets that I responded back with were a great analogy for why batching should be avoided (it had nothing to do with the 140 character limit, it was intentional, honestly.)

I thought that a blog post would allow me to better justify why I tweeted what I had overheard. My primary concern with the statement that was made is that in my opinion, batching all the requested items together to avoid somebody having to rework a plan multiple times slows down the delivery of the highest priority item the effect of which is to deliver value back to the person(s) that requested the work later.

Tying the team up on a larger bodies of work has further affects though. Firstly I think it means that it is less likely that a shorter feedback loop would exist which in turn, would mean that it would be easier for both the team and the person(s) requesting the work to not consider the smallest piece of functionality that could be delivered to meet the requirement. It starves the person responsible for prioritising the work of options to change, it asks for a commitment to deliver all requested items well in advance of their actual start date. Lean and Real Options would suggest deferring your commitment to the last responsible moment and as the InfoQ article I link to, I think this is at the core of being agile.

Working in very small batches isn’t without complication though; the concerns that I am suggesting in the above statements are born out of a bias towards a project based delivery mentality, the alternate to which is a product based one. In the environments that I have worked within it has always been difficult to group products in a meaningful way which in my experience has often lead to product owner groups all of whom need to compete to get their work done which has often ended up leading to frustration. From an engineering perspective, working in smaller increments also increases the need for two things: automated test suites that provide early visibility of breaking changes and also remove the strain on those people that fulfil a testing role in respect of doing the need to do full regression testing. Secondly, an automated release process is needed so that time spent doing releases is reduced.

I should be clear in all of this that I am not suggesting that planning does not add value. The value it brings though is in support of the actual delivery of the requested work. It’s biggest benefit is from a communication perspective and in that respect, it should be something that is recognised as being constantly out of date and therefore that it has a cost associated with it in keeping it up to date.

I mentioned flow specifically in one of my responses and that is point is worthy of expansion. If you imagine a magnet being passed above a bed of iron filings, a weak magnet would have less of an affect on the filings than a magnet with a stronger field would. Similarly small features individually passing through a team’s pipeline have less probability of causing a disruption. In relation to batching, Little’s Law suggests that the more items in a pipeline at any given time has the net effect of slowing (or to use the analogy, disrupting) the other items also in that pipeline. Finally, when you focus on individual features it makes it much easier in my opinion to understand where the bottlenecks are in your system Having the knowledge and then understanding why they exist is the first step in being able to focus efforts on  setting your pipeline up to work as effectively as possible with those bottlenecks.

One Piece Flow, One Piece At a Time

There’s been a lot of interesting conversation regarding the use of Kanban / Continuous Flow Development models and it’s something that I see a great deal of benefit in. I find myself struggling to understand whether it would be possible to start a team that hasn’t really used a process of any description before out with something like Kanban and for them to reap the benefits that people are discussing.

I think the reason I’m struggling a little is that I’ve read nothing that would suggest that Kanban in particular is teaching a team the principles behind the practices and whilst I’m sure that some would disagree, from what I’ve read there isn’t too much focus in that area and given Kanban’s relative infancy, there’s little to speak of about the practices either. For example, whilst I’m sure most could easily see how something like Kanban allows a team to Eliminate Waste, might they miss the fact that the process of mapping your Value Stream allows you to See the Whole (i.e. that your kanban should represent all stages in the life cycle of a feature request)?

At the moment and perhaps given my current circumstance, I feel a lot more comfortable that using a vanilla Scrum implementation will help the teams here at least move towards understanding the principles behind Lean so that we would then be positioned at a later date to implement a flow based model. It was Benjamin Mitchells tweet

Good tips from @henrikkniberg on software processes: evolve the right process for your context, expand your toolkit and experiment!

earlier that prompted me to make this post though I was also reminded of Matt Wynne and Rob Bowley’s presentation at XP Day Evolving from Scrum to Lean. Matt Wynne in particular once posted that for teams, Scrum is The Stabiliser Wheels on Your First Bike and that for organisations, it is like a bulldozer.

Following a suggestion from Rob who attended a presentation by Jeff Sutherland at BT titled Shock Therapy and my later watching a video which is of very similar content, we’re running an experiment here with one of the teams and have got them using Scrum with 1 week iterations.

Scrum is a collection of great tools, and the idea is that when used in this manner, they will help a team very quickly evolve to understand the problems they are facing (retrospectives come in very useful here) and the advantages of working in small batches to best be able to deliver to their definition of done every iteration. It’s easily understood (I accept that to some Kanban may also be easily understood) and some of the agile principles are baked in, it increases communication between the Product Owner(s) and the development team and its iterative nature means that the team is doing this over and over and in doing so, means that the practices and the principles that we’re talking about with them are there for them to see in action, time and time again. I should add that we’re not using Scrum alone, we’re using engineering practices such as TDD, refactoring any of the legacy code we come across and generally focussing in on quality throughout, we’re getting acceptance criteria up front which alone is proving worth the effort.

If the ultimate goal is to move to One Piece Flow then, how specifically is our current use of Scrum helping us?

As I suggest above, there are a number of benefits to Scrum but in respect of doing 1 week iterations, first and foremost, it’s giving both the team and their customers a much greater degree of visibility and better still, it’s happening sooner rather than later.

From the team’s perspective, any of the issues that they are facing are immediately apparent, furthermore and small issues that we might not have caught in say a 2 or 3 week iteration cycle are raised in their importance. The team really need to remove any impediment that they come across as soon as they can to give them as much of a chance as possible of delivering the feature that they have committed to at the end of the iteration. Any problems downstream of the development work going on (e.g. testing, deployment) becomes much more apparent. Doing a retrospective at the end of each iteration and taking meaningful actions away is allowing them to tackle some of the more process based issues they face, they have started to realise too that rather than relying on key people to do repetitive small jobs that come in, they should all be capable of carrying out the task and that furthermore, there would be real advantage in writing a tool so that the job can be done by people outside of the team.

From the business’ perspective, the team is creating a lot of data, quickly. This data includes the team’s velocity, the amount of time a feature that is requested takes to go from being requested to be being delivered and in that we’re also tracking how long that feature is in development.

Whilst I mention above it’s worthy of note again that secondly, it’s helping the team form together as a successful unit, it’s forcing them to self organise. I’ve mentioned before Tuckman’s team development model (I’ve previously heard learning models such as Shu Ha Ri and the Dreyfus model referred to when discussing this advancement.)

Of course, we’ve had a few problems outside of the team too, there are some people that feel that the amount of time that not only do the development team require of their customers in prioritisation sessions and in analysing requirements but that they themselves spend in meetings every week adds too much of an overhead. So far I’ve answered this by saying that the team have some clear goals to meet and once they have done so, they are free to move to whatever they deem appropriate, be that longer iterations or indeed a Continuous Flow Model. I’m also working to help people understand that software development is an inherently collaborative process and that in my opinion, you’ll never see the true efficiency gains to be had from a software team until you change the organisation to accomodate that, something that this post by James Shore alludes to.

Anyway, I’m off to the eXtreme Tuesday Club’s inaugral Kanban Klub meeting tomorrow night so perhaps some of the things that I am struggling with about Kanban will be answered then. In the meantime, Scrum is seeing a little of a renaissance in my estimation as a tool to help development teams quickly mature in to ones that don’t just apply processes and don’t think to ask questions but ones that are questionning for themselves which process that they should be using.

The Greatest Trick The Devil Ever Pulled Was Making People Believe That He Didn’t Exist

James Shore made a great post the other day, Stumbling Through Mediocrity in which he talks about how dysfunctional companies will never truly reap the benefits of an Agile adoption as they aren’t inclined to actually change their ways. Rob Bowley then followed up with his post, Agile Isn’t Enough likening the introduction of agile to dropping a bomb on the internal workings of an organisation which then sends out ripple effects with much wider repercussions that actually get ignored by managers.

I posted a while ago about the fact that I thought that all a team needed to be successful was a clear set of parameters to work within and the support needed to build a learning culture and I maintain that opinion. Indeed when I started at 7digital I was keen not to sell any particular methodology. I set out 3 things that I thought that the department as a whole should focus on; Focus On Quality, Releases as Non Events and Building a Learning Culture. Whenever I spoke about those things I was careful and still am to a certain degree, not to use the dreaded “A” word.

One of the biggest impediments to introducing a change in process whilst I was at my last company, aside from the abundance of people with their own political agenda, was my attempt to introduce “Agile”. I realise now that rather than bestowing the virtues a bunch of practices I should have spent more of my time talking more about the principles. Furthermore, I should have found out whether or not people wanted to change. If they didn’t perceive there to be a need to, then it was my responsibility to find out why, perhaps they had a point and perhaps even in having that conversation they may have heard my point a little more. You live and learn.

What’s interesting though is that whilst there were a large number of people that I used to perceive to be willing to fail, pretty much all of them wanted to get the job done or the project delivered but I was putting it down as a failure because we hadn’t successfully engaged with our Customer, we weren’t in my opinion delivering everything that we could be for them and we hadn’t truly enabled the development team to deliver a quality product. Often I would sight agile as I thought that it would be the answer to these perceived failings and I still think it is, ironically it became a barrier to it though.

Given the posts I mention above, it would suggest that I am not alone in my point of view. My take on it is though, everybody hears the “A” word and thinks “here’s a process that we can use to deliver this piece of software” but actually it won’t. It requires commitment that a lot of people just aren’t willing to give and in many respects, perhaps the word Agile is in fact a hindrance.

Perhaps we should just start sitting with our customers more and hearing exactly what they want from the software that we deliver them and in doing so, put our case across about what we want from them. Do we necessarily have to persuade them of anything other than that we want to deliver the best quality product and that we want to do that by working as closely with them to understand their business model, that we want to as honest and transparent as possible about how we’re progressing. It’s keeping these conversations going throughout the project that’s most difficult. And it’s that very thing that we should be working on the most. The arguments have been won already for how much more successful we are when we use certain techniques as opposed to others, they should be implicit when we say we’re going to develop software.