Joe Blogs

Short rant on Cross Country trains and User Experience

I travelled a couple of times recently on Cross Country trains, and was reminded of some apparent UX fails in the design of their train carriages.

I shouldn’t single out Cross Country for blame on this, as I think these particular carriages made their debut in the early 2000s as Virgin trains.

They have some nice touches that the train I catch down to London is missing – window blinds, coat hooks and personal lights.

But there’s three things I noticed that just don’t seem to have been thought through from the point of view of the user – in this case the passenger.

Electronic display of seat reservations

Unlike paper reservation tickets stuck onto each seat, if you’re travelling without a reservation, and you’re trying to figure out where there’s unreserved seats, you can’t do this easily from the platform.

Even when you’ve boarded the train you have to wait at each row while the seat reservations scroll across the tiny screen. Plus you’d have to be pretty tall for this to be anywhere near eye-level.

Okay, so the electronic display probably saves time and money for the train operator, but I think this is at the expense of the user experience. With a paper reservation slips in each seat, you could see at a glance where unreserved seats where.

And what happens when the electronic system fails? Musical chairs as everyone grabs a free seat, until the passenger with the reservation ticket comes along.

The ceiling

The ceiling of the carriage isn’t a straightforward concave curve. Instead the ceiling includes a convex curve. Presumably this is aesthetically pleasing.

However it also means that the luggage space above the seats is smaller, and passengers can’t fit as much in. This seems like style over substance.

Open Sesame

The button to open and close the door between the carriage and the ‘vestibule’ is ON the door. When you approach the closed door this seems fine, as the button is right in front of you.

But when the open door starts to close, you’re trying to hit a moving target. Many times I’ve seen people queuing to get into a carriage (because those in front of them are peering at the electronic displays, or trying to cram their luggage into the undersized overhead luggage space) getting squashed by the doors because they can’t figure out where the button is.

Even R2D2 might’ve struggled to stop the garbage compacter if the inch-wide UI to do so had been a moving target…

This seems poor as you’re making your user interface, to operate something as fundamental as a door, inconsistent. Plus it’s really not great for people with poor motor skills.

To top it all off, the door-mounted button is accompanied by a sign warning you not to place your hands on the door.

I just thought of another one, which others have written about too…

Electronic toilet doors

So you press a button to open the toilet door, you enter and press a button to close it, then you have to press another button to ensure its locked.

What a faff. I mean, it’s a door, why does it need to be this complicated? Sure, there are instruction in the loo to ensure you lock it before dropping your trousers, but should you need instructions to operate a door?

What happens when the LED for the locked indicator fails. How can you be confident the door is locked?

What happens when the power fails? Do all the toilet doors just open, revealing some poor passengers mid-poo? Or do they all just lock, leaving them stuck in a stinky toilet all the way to Aberdeen?

A friend walked in on a guy in one of those toilets once, although she suspected from the grin on his face that he’d deliberately left it unlocked… urgh.

Usability Testing?

I’d be really interested in what kind of usability testing was done on these types of train carriage when they were designed.

Maybe this comes across as a bit of a rant, but it seems that the basic user experience for passengers aboard these trains could’ve been improved by a few simple changes.

Image Credit
“Class 390 Interior” by Sunil060902 – Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.

Stubbing API calls using stub.by

Typically, if I was developing an application that talks to an API, the TDD approach I’ve historically followed would end up with me using a mocking framework to stub a service component that talks to the API, in my unit tests. I’d then include this service in my implementation code via dependency injection. A fairly common approach of writing tests and stubbing dependencies I guess.

WebMock and Stubby

Colleagues of mine who build rails apps often use the WebMock gem to ‘catch’ API requests and return a specific known response – a different kind of stub.

In the spirit of @b_seven_e‘s DDDNorth 2014 talk at the weekend – of our ruby and .NET teams learning from each other – I found a .NET package called Stubby which does a similar thing to WebMock.

The source code and documentation can also be found on github – https://github.com/mrak/stubby4net

Getting Started

To try Stubby out, I’m building a simple ASP.Net MVC web app, that will get it’s data from a ReSTful API.

I began by adding the Stubby NuGet package

PM> Install-Package stubby

So to try out Stubby I began with a single ‘happy path’ test that tests – given an API returns some data – if my web application returns some data.

[Test] public void When_you_view_a_muppet_profile_then_you_see_the_muppet_name() { var controller = new MuppetsController();
var actionResult = controller.GetMuppet("gonzo");
var viewModel = (MuppetViewModel)((ViewResult)actionResult).Model;
Assert.That( viewModel.Name, Is.EqualTo("Gonzo")); }

This seems straightforward, but where is the actual stub for the API I’m going to call..? This is set up via the following code in the test fixture –

private readonly Stubby _stubby = new Stubby(new Arguments { Data = "../../endpoints.yaml" });
[TestFixtureSetUp] public void TestFixtureSetUp() { _stubby.Start();}
[TestFixtureTearDown] public void TestFixtureTearDown() { _stubby.Stop(); }

The YAML file which defines the actual behaviour of the stubbed API looks something like this –

- request: url: /muppets/gonzo/?$ response: status: 200 headers: content-type: application/json body: >{ "name":"Gonzo", "gender":"Male", "firstAppearance":"1976" }

This is a basic request/response pair. In this case it matches any GET request to muppets/gonzo and returns a 200 response with the specified headers and body.

There is much more you can do in terms of defining requests and responses – all detailed in the Stubby documentation. You can also set up the request/response directly in code, but I went with the YAML file approach first.

When I run my test – Stubby starts running on port 8882 by default. I add some config to my test project to ensure that my implementation points at this host when making API calls, and then every API call that is made will be caught by Stubby. If I make a call to /muppets/gonzo then this will be matched against the YAML file and the response above is returned.

So now I have this failing test, so I can go and write some basic implementation code to make it pass. In my case I add some code to the controller which makes an API call, de-serialises the JSON returned into an object, and then maps this to a ViewModel which is returned with a View.

More Tests

Once I had this test passing I extended my API stub to include the scenarios where the API returns a 404 or a 500 status code.

- request: url: /muppets/bungle/?$ response: status: 404
- request: url: /muppets/kermit/?$ response: status: 500

This allowed me to explore how my application would respond if the API was unavailable, or if it was available but returned no resource. In this case I decided that I wanted my application to act in different ways in these two different scenarios.

Refactoring & Design

With these green tests, it now feels like past time to refactor the implementation code.

I haven’t ended up with the service and repository components that I might normally end up with if I’d followed my old TDD approach of writing a test that stubbed the API component in code.

I can put these components in myself now, but it feels like I am a lot more free to exercise some design over my code at this point.

This feels like a good thing. I have a set of tests that stub the external dependency, and give me confidence that my application is working correctly. But the tests don’t influence the implementation in any way, nor do they mirror the implementation in the way that you sometimes get with my previous approach. The tests feel more loosely-coupled from my implementation.

This also feels a bit more like the approach outlined by @ICooper in his “TDD – Where did it all go wrong” video – stub the components which you don’t have control over, make the test pass in the simplest way possible, then introduce design at the refactoring stage – without adding any more new tests.

Testing Timeouts

Another interesting thing we can do with Stubby is to test what happens if the API times out. If I set up a request/response pair in my YAML file that looks like this

- request: url: /muppets/beaker/?$ response: status: 200 latency: 101000

this will ensure that the API responds after a delay of 1:41 – 1 second longer than the standard HttpClient timeout.

So by requesting this URL, I can effectively test how my application reacts to API timeouts. Obviously I wouldn’t want to run lots of timeout tests all the time, but it could run as part of an overnight test run.

This gets more useful if I’m creating an application which makes multiple API calls for each screen, and I want my application to fail more gracefully than just displaying an error page if an individual API error occurs.

Is this a worthwhile approach?

Well it feels pretty good, so after discussion with my colleagues, we’re going to try it out on some production work, and see how it goes… watch this space…

Failure IS an option

I wrote this post after coming off a daily ‘stand-up’ call where one Developer admitted he “didn’t know what he was doing” because he’s covering for one of our UI Developers while she’s off , and a DBA told us we weren’t seeing the data we expected that morning because he ran the wrong package by mistake.

It got me thinking about how it’s important to encourage an atmosphere where people aren’t afraid to talk about the mistakes they’ve made.

No-one admonished these people for saying these things. We respect someone for being open about the fact that they made a mistake, and then fixed it. We admire someone for actively stepping out of their comfort zone to work on something they’re not used to – it broadens their skills and reduces the number of single points of failure in our team – which in turn helps to keep our work flowing.

Whilst failure can be bad – it is also a chance to learn, and improve. It’s okay to make mistakes, and to admit when you don’t know the best way to do something, as long as you learn from it!

photo credit: ncc_badiey cc

10% time hall of fame

I wrote a post last year about introducing 10% time in our workplace, and the twitter feedback idea that I started working on.

A few months ago we had our first entry into the 10% time hall of fame. This is a coveted position reserved only for those ideas that have been developed and iterated to the point where they’ve been released into our main live service.

Read more about Michael Cheung’s 10% time project – Optimise Health A-Z for small viewports.

Hopefully this will be the first of many ideas to start out as 10% time projects, and make it all the way into our live service…

But why no technical stories?

In my current workplace we’ve been using User Stories in various guises for a while now. One of the things that frequently crops up is whether these Stories can or should be technical or not?

To start with it may be useful to remind ourselves of some of the aspects of a user story…

A User Story describes something that a user needs to do, and the reason they need to do this. They are always written from a user’s point of view, so that they represent some value to the user, rather than a technical deliverable.

They represent the who, what and why – an example might be –

As an expectant parent
I want to receive emails about parenting
So that I can read information about how to best care for my child

They are intentionally brief, so as to encourage further conversation, during which the needs of the user can be explored further, and potential solutions discussed. They are not intended to be detailed up-front specifications – the detail comes out of the conversations.

So who owns the User Stories?

User stories are designed to represent a user need. Most of the time these users are members of the public, but we don’t have our actual users in the office writing and prioritising stories, so we have a proxy for them instead – which we call the Product Lead (PL).

Part of the PL’s job is to represent what our users need – they use User Stories to capture these needs, ready for future discussion. So the PL owns the stories and their relative priority. If this is the case, then the PL needs to understand the stories, so that they can own them. If the backlog has technical stories in it, then it is difficult for them to prioritise these against other user needs.

For example, if the PL sees a story about enabling the public to search for GPs, and another story about reconfiguring a data access layer – they’re likely to prioritise the user-focussed story as they can see the tangible benefit. The technical work is totally valid, but it needs to be derived from a User Story – everything should start with a user need.

What if it’s an internal user?

As suggested above – most of the time the users we are capturing the needs of, are members of the public. Sometimes we use more specific personas in order to capture the needs of specific groups of people e.g. expectant parents

Other times, the users are our own internal users. We can still express their needs in terms of a user story –

As a data manager in the Data Workstream
I want to configure search results views
So that I can change the information displayed in accordance with DH policy

Here there is still an underlying need of the public, but the story is expressed from the point of view of the Data Manager. We could have just written down a technical story like –

Build a data configuration system for results views

but if we do this we have skipped a step. We have assumed that we know the single best solution straight away. Maybe there are other options to achieve their initial need – maybe if we stop to think about these other options we will find one that is cheaper/better/faster too.

When should we do technical design?

Although we need to some technical design up-front in order to set a general direction of travel, the detailed design work for a particular story should be done as close to actually implementing the story as possible.

In the past we have suffered from doing lots of detailed design of technical implementation well in advance of actually being ready to deliver that piece of work. Often by the time it came to deliver that piece of work our understanding had evolved and the up-front design was no longer valid.

Don’t try to do the technical design work when you first create the story. Wait until we are ready to deliver that story, and then look at the technical options available. By doing this work Just-In-Time we are much less likely to waste effort thinking about a solution that will never be delivered.

How do we track progress?

We track progress in terms of completed stories. If we keep those as User Stories then we are measuring our progress in terms of actual value delivered to our users.

If we break stories down into lots of technical stories then it may look like we are making lots of progress, and that we are very busy, but we could have delivered very little genuine value at the end of it all. If, for example we reported that –

“We’ve completed 90% of the business layer code”

that sounds very positive, but we could have delivered no actual working tested functionality for our users at this point. By keeping our stories user-focussed, our progress is also measured in terms of value delivered to users.

How do we get from User stories to technical scope

We’ve talked about how important it is to start with user needs, but ultimately we need to build something, so we have to get down to the technical detail at some point.

One way of ensuring that all scope maps back to an overall goal is to use a technique called Impact Mapping. We used this on a very technical project to ensure that all of the technical deliverables mapped back to an overall goal.

At the same time as deriving those initial stories, we’d usually be thinking about the overall technical approach. We’d look for answers to high-level questions around what technologies to use, and what approach to use. These wouldn’t be technical stories though – this would likely be documented in a lightweight high-level design document.

Story Decomposition and Technical Tasks

Once we’ve derived some initial user stories from our goal, we’d continue to break those stories down until we arrive at the technical scope.

User stories can be split into smaller stories but we always try to retain value to the user in each story, rather than making them technical.

For example, the story above about parenting emails might be split into smaller stories like –

As an expectant parent
I want to sign up for email notifications
So that I receive useful information about caring for my baby

As an expectant parent
I want my email address to be validated
So that it is clear when I have entered an invalid email address

As an expectant parent
I want to provide my first name when signing up
So that the emails I receive are personally addressed to me

Each of these stories is a smaller deliverable, but still makes sense from a user’s point of view.

Further to that, once we end up with nice small stories, we can create a list of technical tasks. Each story might contain the tasks needed to deliver that particular story. The tasks get down to the level of technical detail around what components and packages need to be altered, in order to deliver.

Ultimately – we will end up with technical pieces of work to do. Key is that all of these are derived from user needs.

* We don’t have to use User Stories for EVERYTHING

Okay, so we go on about User Stories a lot, but ultimately they’re just a tool for communication. A User Story represents some change we want to make in order to deliver some value. It’s a cue to go and have a further conversation about that piece of work, and that value.

If we can have these conversations, and deliver the value, without writing stories every time, then maybe that’s okay. The most important thing is for the people concerned to talk to each other frequently, deliver the work in really small chunks, and get feedback as often as possible.

User stories do really help us with this, but there might be occasions when they’re not the best tool…

Conclusions

Yes, we’re going to end up with stories that are technical in their implementation, but it’s important to not jump straight into that implementation. Think about our users, and their behaviours – and derive the stories from that.

Friends & Family Tweet

For my 10% time project I have decided to look at whether Friends and Family Ratings could be submitted via Twitter.

The idea was that if you’re sat in a GP waiting room you might be tweeting, and if you were shown a particular hashtag(s) then you could use these to tweet a view on the service you’re receiving.

This example could be flawed in several ways, as you’re more likely to want to give an opinion after you’ve actually seen the doctor, not while you’re waiting there…

However I’m going to explore the possibilities and see where it takes us – there may be mileage in integrating with Care Connect so that people can submit real-time ratings, comments or complaints via Twitter. It could be extended beyond Twitter itself to other channels. We could make use of geo-location (thanks Russell) or QR codes (thanks Sean) to enhance the experience.

EDIT – there seems to be appetite for this today
http://www.bbc.co.uk/news/health-24669382

The technical bit

So far I have a basic Gherkin spec which looks like this –

Feature: Grab Tweets By Hashtag
In order to store tweets for processing
Twit can grab tweets with a specified hashtag from twitter

Scenario: Grab a new tweet with a hashtag
Given there is a new tweet with the hashtag #nhsfft
When you wait 15 seconds
And you run Twit
Then this tweet should be stored

In the background this automatically posts a tweet from a personal dev account I set up, and then runs a console app (called Twit, right now) which searches twitter for tweets with this hashtag and then stores the tweets in a RavenDB store for future processing. The 15 second pause is there as the tweet wasn’t showing up in search results immediately – this sucks a bit as it slows down the automated test…

The next step is to write a separate process, which takes these stored tweets and posts them to Choices via our comment capture API.

I’m trying to practice what I preach (!) and use a BDD and TDD approach to the coding, with lots of refactoring as I go to ensure the code-base stays simple and clean.

It’s also good for learning – I found that you don’t have to store the NuGet packages in source control (thanks Steve), and am learning bits and pieces about RavenDB too.

Interacting with Twitter poses some challenges too. I began writing raw requests, but found dealing with OAuth was eating up time (when I just want to get something working) so am now using a couple of .Net components that wrap around the Twitter API. It’s likely that I’ll have to move from using the REST API to the Streaming API too, in order to ensure we don’t miss any tweets.

I’m also learning more about git – it’s only in a local repository at present, but will hopefully be in github soon!

10% time

We just launched a new initiative in our team to encourage more innovation and learning. Our 10% time is a bit of a cross between Google’s 20% time, and Dragons’ Den.

The idea is that any team members in Product Delivery can spend Friday afternoons working on their own project. It can be anything they want, but our panel of Dragons must agree to ‘invest’ a certain number of afternoons in the idea first.

Pitch

To get involved – team members deliver a short pitch explaining

what the idea is
how it will benefit the wider organisation
how long they initially need to work on it

Investment

The Dragons can choose to invest a certain number of afternoons in the idea, or ask for more refinements to the idea. When the end of the invested time comes – we’ll have demos of the different ideas, to see what we’ve come up with.

Ideas

There are some already some interesting ideas being pitched – ranging from a historic twitter analysis and a pregnancy care planner app, to a virtual fridge. Some people have pitched individually, and others have worked in pairs. The methodologies, technologies and tools used to build these ideas are entirely up the individuals building them, so it’s a chance to try out some new stuff.

Sizing, Estimation and Forecasting

The story so far

Over the last few years we’ve tried a variety of estimation and planning techniques. We’ve suffered from our fair share of Estimation Anti-patterns and tried various approaches to avoid these.

I thought it’d be useful to outline some of the approaches we’ve tried, the problems we’ve encountered, and how we’ve reacted to those in order to get to where we are now.

2010

Back in 2010 estimates were forced to fit a previously agreed plan:

“What’s the estimate”

“60 days”

“It needs to be 30, go away and re-estimate it”

This is a cross between the Target Estimation and Comedy-driven Estimation anti-patterns, and obviously it’s just a big farce – what’s the point in estimating in the first place if you’re just going to have a fixed time, scope and resource all imposed on you.

This approach led to teams and individuals being put under a great deal of pressure, and generated bad feeling between the people who imposed the ‘estimates’ and those who had to stick to them.

Of course corners were cut in order to meet the fixed estimates, which led to further technical debt, which just exacerbated the whole problem for future projects – the Done-driven estimation anti-pattern.

2011

During 2011 we gradually moved away from ‘fixed’ estimates. We introduced a few fairly standard ideas –

Estimating in ideal days

We started estimating in ideal days, to take into account of the fact that a Developer doesn’t get to spend their entire day dedicated to the estimated item that they’re currently working on.

This worked okay, once we finally hammered out the exact definition of an ideal day…

“Does an ideal day include meetings?”

“But what if the meeting relates to the story they’re working on?”

Having the people who are going to do the work doing the estimation

We tried to throw out the idea that a single individual could estimate a project more accurately and precisely than the developers who were familiar with the codebase, and who were about to do the work.

Estimates would still be questioned by people who weren’t going to do the work. We’d get Architects or Managers questioning why Developers thought something would take, for example, 3 days –

“That story’s just a few lines of code isn’t it”

This was frustrating, and we probably did waste more than a few hours justifying estimates to people outside the team.

Planning poker to derive estimates from the group, not individuals

The introduction of planning poker was quite good fun to start with. It bought the team together and helped to alleviate some of the discussions and justification that we had to go through.

However, it sometimes did feel like a bit like a negotiation – with some people deliberately going in low to try to bring an estimate down.

Velocity – planning based on past performance

We introduced the standard idea of velocity from Scrum –

Take the number of ideal days you complete in an iteration, and then plan your next iterations based on that.

This was sound, but unfortunately it was described by whoever sold the concept to senior management as being a percentage measure. So if a team got 30 ideal days of stories completed in an iteration of 40 elapsed developer-days, the team had achieved a ‘75% velocity’ – this was really ugly, and came to hurt us, as you’ll read below.

We struggled a bit with the idea of the team committing to a sprint goal. There were a lot of dependencies on other teams that we just didn’t account for, so we could never really meet what felt like reasonable goals.

Relative Estimates

We started to estimate work based on it’s relative size, compared to work we’d done previously. After all, this seemed like the quickest and generally most reliable way to estimate. If you ask a decorator to quote for painting a room, they can usually give you a rough quote without measuring up, because they’ve already painted lots of rooms of roughly the same size before.

This approach for us was pretty successful – if we’d tackled a similar size project in the same product area, we could look up the actual effort we expended on the previous project and use that to guide our estimate for the new project. When the newer project was complete, looking at the actuals showed us that this was a fairly accurate method.

It helped us to resolve the Fractal Estimation anti-pattern that we’d suffered from in the past, because we were now looking at sizing the project as a whole to start with, as opposed to trying to break it up and estimate each constituent part.

The problem was when we had to estimate something that wasn’t really similar to anything we’d built before.

Overall things improved during 2011 – the people doing the work had more control, and we had a method by which to size things, and plan work. But then things started to unravel…

2012

It gradually became clear that some of the things that we thought were working, weren’t really…

Story Points

The business didn’t understand the concept of Ideal days, so we re-branded them as Story Points, where a story point equates to an ideal day. This didn’t really help though as we never built a shared understanding that Story Points are a relative measure of size, as opposed to an exact measure of time taken to do something.

“How big is the project?”

“30 points”

“You have five developers, so it’ll be done in six days?”

“Erm…maybe…”

What’s velocity?

The concept of Velocity was never been well understood by the business either. It became seen as a measure of efficiency, or utilisation. To paraphrase:

Managing Director: “What’s velocity?”

Programme Manager: “It’s the time that developers aren’t working – like when they go for lunch or a p*ss”

and so the percentage thing came back to bite us – velocity was used as a stick to beat the teams with –

“The Developers are only working at 60%, we need to get them to work at 70%”

Targets

We moved away from planning based on past performance and trying to improve on that, to planning based on fixed targets per developer. The Target Estimation anti-pattern again.

To increase speed; targets were set for developers to develop a certain amount of work each week.

The planning was based on one big resource pool of developers (only), with individual targets aggregating up into one giant target.

The focus was on individual developer productivity rather than actual throughput of developed and tested stories. This led to a bad working environment, much frustration, and undesirable behaviours.

Some teams adopted the Velocity-driven estimation anti-pattern in order to get around the targets they were set. But it didn’t mean they were delivering any more work – it just meant that Story Points became even more meaningless…

Budgets

A positive thing we introduced in 2012 was the idea of budgets for pieces of work. This was the starting point for turning the question around and establishing what each piece of work is worth to the business –

“How long will this project take you?”

“We’re not sure yet. How long would you like us to spend working on it?”

Developers-only

As you’ll have picked up from the story so far – the vast majority of the focus was on Developers, and only Developers. They were widely regarded as the limiting ‘golden’ resource, and as such theirs was the only work that needed estimating – everything else that needed to be done like story-writing, deployment and testing would just fall into place.

This is partly the Done-done-driven Estimation anti-pattern. The problem with focussing on just Developers is that they cannot deliver work in isolation. There are many inter-dependencies on other roles such as BAs, EAs, Testers, Infrastructure, DBAs and so on.

It is the team that delivers work, not individuals. You can try to estimate the effort that a Developer alone will have to put in to deliver a story, but that really is only a part of the work needed to deliver end-to-end.

2013

As part of the more focussed agile transformation process, we decided to have a complete re-think about how we estimate and plan at the team-level.

Principles

We came up with some principles by which we wanted to base our estimation and planning. These are based on the experience of the team, and tied in with the feedback that we received from some external consultants we were working with.

Plan based on past performance
Track the whole cycle, not just development
Estimates are not exact quotes
Plan at a team level and scale up, not the other way around
Limit work in progress
Separate the methodologies used for planning, from that used for performance management

What matters

We considered having another crack at using story point estimation and velocity as it was intended, but decided that there were already too many misconceptions around this for it to be a success.

Instead we opted to try some of the more empirical techniques associated with Kanban, which tied in nicely with our move away from iterations to more of a flow-based delivery model.

The beauty of these techniques is that they focus on what matters – the question that our colleagues and management want an answer to is generally

“When will we get this product?”

not

“How much effort will it take?”

We started focussing on the elapsed time that it took to deliver things, as opposed to how much effort a particular role puts in to get it there.

Efficiency

An eye-opening aspect of this is to look at Business Process Efficiency (BPE) – which is the ratio of the time that a piece of work is actively being worked on (by anyone), to the total time that it takes to deliver that piece of work.

Many organisation are working with a typical BPE of just 15%. So for the vast majority of the time it takes to deliver something, that thing is just sat waiting to be worked on – perhaps at a handover between roles or teams. So all the work we put in to estimate effort was really only focussed on a very small portion of the time it takes to deliver – and focussing on developers only magnified this even more!

The here and now

Flow and Forecasting

Where we are now is that the teams aim to split stories up nice and small. They then count the number of stories in each state of their kanban system each day. We use this to track each teams’ flow.

We generate Cumulative Flow Diagrams (CFD) and record team throughput. Both of these can be used to forecast future delivery. The great part is that this is not based on anyone’s judgement of the size of a piece of work – it is based on the actual empirical figures for how long it takes to deliver.

Cycle Time

We track the Cycle Time for stories – this is the time it takes to deliver a story end-to-end. It is currently surprisingly high, and we’re challenging teams to see what they can do to reduce their cycle times – the quickest win for this is to reduce the time that stories sit in a particular state waiting for someone to pull them into the next. We can improve on this by limiting the number of things that we work on at any one time.

Sizing

When we set out with this method of using empirical data to forecast, instead of estimating, we were concerned about the disparity in the size of stories. If we’re just counting stories what would happen if we delivered all of the smaller stories first, and were left with all the bigger ones – it’d look like we were way further head than we really were.

To counter this we sized stories small, medium or large. We had one person per team doing this to generate some consistency, and it was a quick process that was done as part of the story’s refinement.

We then tracked CFDs for both story count, and a kind of ‘weighted count’ that took the relative size into account e.g. a medium is twice a small, and a large is twice a medium.

So this took differences in story size into account, but what we found was that over time the slope of the CFD’s accepted state was roughly the same for the weighted and non-weighted count. A forecast based on story count alone should be as accurate as the forecast that takes story size into account.

For this reason, we’ve stopped sizing altogether and now just count stories. What’s key is that we aim to get a reasonable consistency of small stories.

Time-boxes

Back when we introduced budgets we started to turn around the question of how long something would take, to how long did the business want us to spend on it – what is it worth?

We’ve extended this to reinforce the idea of fixing time and flexing scope, by planning time-boxes. A project has an assigned delivery time-box during which the team pull stories from the backlog for that project. Once that time-box is over the team finish any unfinished stories off, but start pulling new stories from the next project time-box to which they’re assigned. Essentially the time-box controls which project the team pull new stories from – or in Kanban terms – where they replenish their system from.

Project cycle-time

The question that remains is what is a reasonable length of time-box to plan in for a project.

At a higher-level what we need to do next is start tracking the cycle time of overall projects. We can then use this to plan sensible time-boxes for delivery of future projects of a similar nature.

The Future

It’s been a long and sometimes frustrating journey – but it feels like we are now in a better place. We now spend a lot less time sizing and estimating things – practically none in fact.

In future we aim to widen the gathering of metrics look for further patterns to see what impacts on delivery. There are still challenges ahead as we embark on newer, bigger pieces of work, but I think we are better equipped to give honest, accurate forecasts of what can be delivered, and by when.

PS. If you’d asked me to estimate how long it’d take me to write this blog post I’d have said a couple of days. It took a bit longer…

photo credit: lemonad via photopin cc

photo credit: eatmorechips cc

photo credit: bensutherland cc

What does a Product Lead do?

The Product Lead (PL) role has existed in our organisation for some time. On the face of it, the role is broadly comparable to the Product Owner (PO) role, as described by the Scrum agile methodology – some of the responsibilities match up.

The main difference is that the PO role is generally regarded as a full-time role working very closely with the delivery team, whereas our PLs are almost always individuals who have other full-time roles within their respective workstreams.

This poses problems, and creates frustration, as we have competing priorities for our PLs’ time. We could try to enforce the traditional PO role, but really it is for us as an organisation to define our roles, and what they do.

In an attempt to clarify the role, we wrote this guide for our Product Leads, around the things that we expect them to do –

Vision and Goal

You’ll be outlining the vision for the product or feature, and explaining the overall goal. Ideally this goal would be measurable. If you’re not sure what the overall goal is, try asking yourself ‘Why?’ until you reach the real value – see The 5 Whys

Detail

You’ll work with the delivery team to help define the features we’re going to build. We do this through User Stories and their Acceptance Criteria. The best way to do this is to provide us with specific examples – lots of them.
e.g. How should it behave when we do this? What should users be able to do in this scenario?
We have BAs to help with this, but you need to own the backlog of User Stories that will be built.

Value and ROI

You’ll need to justify your product, in order to obtain a project budget. Once we start working on your product or feature, you’re responsible for guiding the team to build the right thing, that delivers value and meets the goal you’ve defined. Any information you can provide on what impact or benefit the product has had once it’s been released is useful too.

Prioritise

As we’re working within a limited project budget, its important that we build things in the right order. We don’t want to use the budget up working on non-critical features. The best way to present priorities is a numbered list in priority order. You own the prioritisation, and you can decide what we should build next, as we learn more about what our users need.

Verification

We need you to get involved in verifying that we’ve built the right thing, as early as possible. The testing doesn’t all wait until the end any more. We can provide you with a link so that you can see your product growing in a test environment on a daily basis. The more closely involved you are with verifying the product, the better you will be able to prioritise future work, and shape the product.

Feedback

We love feedback! Especially from real users. It’s what enables us to build better products. The weekly review is a great opportunity to provide feedback on what is being built.

Meetings

We have a number of meetings during the development of your product. First of all you’ll need to work regularly with BAs, Architects and UX Designers in coming up with some early ideas and design work. Once the development and testing work kicks off we have daily stand-ups, weekly Reviews and Planning sessions, and Retrospectives where we look at how things are going. You need to be involved in as much of this as possible.

Team

Whilst your product is being designed and built, we’d like you to be part of the Delivery Team. We know you all have busy jobs to do, but the best deliveries happen when the Product Lead gets closely involved.

Proxy

You must represent other stakeholders in the business, including your Managers. Make sure you manage their expectations, and let them know about any decisions that are being made, as early as possible. You are the main line of communication between stakeholders and the team delivering the product.

Documentation

You’ll need to update the documentation for your product in our Product Wikis. This involves creating and updating wiki pages so that the relevant people across the organisation understand, and can support, your new product or feature.

Impact Mapping on ODS

Impact Mapping is a technique for deriving scope from goals. It’s useful for looking at different options for meeting a particular measurable goal.

I’d been keen to try impact-mapping-cover it out since picking up a copy of Gojko’s book on the subject at BDDX2012. I don’t think we’ve used it ‘end-to-end’ yet – to truly evaluate and measure different options – but I have found it really useful in deriving scope and ensuring that scope maps back to an original goal.

A good example of where we used Impact Mapping to derive User Stories on a very technical project was the ODS migration project. Due to the restructure of the NHS – removal of SHAs and PCTs, and introduction of CCGs and ATs – we had to do some work in order to reflect this structure to the public.

This involved extensive changes to multiple ETL packages, and changes to the code to generate organisation profiles. We could easily have dived straight in to the technical detail and decided which ETL packages to change.

Instead we went through an exercise to ensure that all of the technical work we were to undertake was mapped right back to the overall goal.

1. Goal

Firstly we established the overall goal that we’re working towards. In this case the goal is for the nhs.uk site to accurately reflect the new NHS organisation structure.

Normally with this technique we’d want the goal to be measurable – like an increase in profit of 5%, or an increase of 10000 subscribers. In this case the goal is effectively binary – either the site does reflect the new structure, or it doesn’t…

This is the cell at the centre of the map.

* In hindsight maybe we could have been more rigorous with the goal and measured it by surveying site visitors as to their understanding of the new structures. This may have been a better measure of the true overall goal – which is to communicate the new structure.

2. Actors

Next we went through an exercise of mapping out the different users, or actors associated with the goal of reflecting the new NHS structure, for example –

The external organisations who provide data feeds into nhs.uk – ODS, PPD, EDOS, PCIS
Organisation profile content managers
The general public themselves, as they are going to view the new organisation profiles on the site.

The actors are represented by the pink cells surrounding the goal. ‘Big Shots’ is a role that my colleague Ashish came up with – the senior stakeholders who use the accountability views in our ‘Find & Compare’ product.

In future I’d like to identify more specific roles, and possibly tie them in with the personas that our UX team use.

3. Impacts

After identifying these different roles, we looked at the different ways in which they contribute to reflecting the new NHS structure, or how their behaviour will change as a result of the new structures. What are the impacts we want to have on the actors, or the impacts that we need the actors to make.

The blue cells represent the impact. The map really starts to expand at this point as some roles can do many things to bring us towards our goal.

4. Scope

Once we’d identified all of these behaviours we could then start to derive the scope required to enable that behaviour to occur.

So for example – ODS provide a nightly feed of the new NHS organisations, this is how they contribute to our goal of reflecting the new NHS structure. The work we need to do to enable that is to create a series of nightly import and data-synchronisation processes.

These can be derived as a series of user stories – something like

In order that nhs.uk can reflect the new NHS organisational structure
ODS can provide a nightly feed of CCG data

Each story is represented by a green cell. The overall map looks like this –

Some of these stories were high-level and were then broken down further – we just added another level of nesting of green cells to represent this break-down.

Visualising the backlog

I’ve found the map a really useful alternative way of presenting the product backlog for a particular project or piece of work.

Since we started using User Stories to represent requirements, some have commented that in doing so we lose sight of the bigger picture.

One thing a colleague mentioned the other day is that User Stories are like leaves on a tree, but when we put them into the backlog it’s like stuffing all the leaves into a bin bag – making it pretty difficult to maintain any context between them.

Now with our Impact Map we’ve re-formed the tree with the leaves in place. We can stick our map up on the wall, and cross items off as they’re completed, or make notes against them.

Living Document

We had a good stab at capturing the necessary scope up-front, but naturally things emerged as we progressed with the work. Having this Impact Map meant that we could always check where suggested scope fitted into the big picture – how did it help us get towards our goal.

“Does it help us meet the goal?”

Some technical work was suggested around changing the mechanism by which we import certain data feeds, but when we examined it in the context of the map, it turned out we didn’t really need to do it. If we had jumped straight into the technical work without looking at the users, their needs and impacts, we would have wasted time doing this unnecessary work.

Tracking

I was acting as BA on this project, and I personally found the map a really useful way of tracking scope, progress towards our goal, and what our priorities were. The map itself was useful during planning, prioritisation and design sessions, as we had a visual representation of the scope in front of us.

Each green ‘scope’ cell was set up to link to our work tracking system, and it was easy to visually represent progress on the map by ticking off or shading the completed stories. This showed how much closer we were getting to our overall goal.

The map was also used to show up blockers and dependencies in scope. We used icons in Freemind to indicate key questions and blockers that arose during delivery.

Tools

For the map described in this post I used FreeMind. Once you have a few keyboard short-cuts set up it’s quick to colour-code the cells and add icons and hyperlinks in.

I also tried out a MindMup version too.

Next steps…

The big thing I felt was missing from this first attempt at Impact Mapping was making the goal measurable. When I first created the map I didn’t think there was much in the way of metrics that we could use to measure our progress towards the goal – there’s certainly not a monetary value from our point of view. We could look at measuring the amount of traffic that certain areas of the site receive before and after implementation, and we could measure the number of service desk calls we receive in relation to the NHS structural changes.

This is something we need to work on – tying the features we deliver back to explicit, measurable goals. I think it really helps the whole delivery team understand what we’re aiming for, and if we can keep our options open for meeting that goal, and measure our progress towards it, then all the better.

We’ll keep refining our use of this technique on further product developments. We’ll push to get the Impact Map created as early on as possible, when we’re still figuring out the overall goal.