product – Joe Blogs

Product management takeaways

I’ve just spent two days with two Jeffs (Gothelf and Patton). They ran a great product management course, and I wanted to quickly get down some of my personal takeaway points, while it’s fresh in my mind.

The term MVP has been wrecked

This rang true as one of my bugbears. I’ve always referred to the definition from Eric Ries’ 2011 book The Lean Startup:

The smallest thing you can make or do, to test a hypothesis

But I hadn’t realised the phrase was originally coined in 2001, by Frank Robinson, as:

The smallest product to meet it’s desired market outcomes

I can see why these clash, and how the wording of Ries’ definition causes confusion.

The term has become overloaded where I work, often being used to refer to:

The scope that’s left once we’ve fixed the people working on the team, and the timeframe they have available.

I’m aiming to use the two alternative terms that Jeff Patton suggested, to be more specific in referring to the Robinson or Ries definition, respectively:

Smallest Successful Release

Next Best Test

Defining metrics for impacts and outcomes

This is pretty much what we’ve been doing on our work around self-referral into psychological therapies; figuring out what the key impact is, quantifying that, and then defining measurable outcomes that are leading indicators of us achieving that impact. We’ll do more on visualising this.

Hypotheses are going to hang around

I’ve used hypotheses to track user research experiments and activities, but I think a mistake we’ve made is to think that a hypothesis is done with, either validated or not, after our first experiment.

It’s fine to run a succession of experiments against a given hypothesis, each experiment becoming higher fidelity, as we learn more (see Giff Constable’s Truth Curve).

A hypothesis likely won’t be completely proven until we get something into production at scale. The earlier, lower-fidelity experiments allow us to stop earlier if a hypothesis is disproven.

Visualising multiple backlogs

We’ve tried before to visualise the end-to-end flow of work, from idea through to delivered user stories, and ended up ditching it in favour of a much simpler ‘Todo, Doing, Done’ type of board, for the whole team.

Jeff Patton showed some examples of separate backlogs and boards, for different types of work. I think we should have another go at this, embrace the idea that there are different types of Discovery and Delivery work going on, and visualise accordingly.

This also ties in with the ideas around dual-track development, another thing we could get more rigorous with. There were some great ideas around adapting your ‘scrum’ sessions to play towards these two different types of work going on.

Three ways of prioritising, for three different situations

These different types of work, and separate backlogs, should be managed and prioritised in different ways too:

New opportunities identified should be prioritised against our overall strategy or vision, to ensure we engage with things that keep us heading in that direction.

For Discovery activities we should prioritise based on what we need to learn the most about; assessing hypotheses based on risk and potential value.

Once we, as Product Managers, have defined a minimum successful release, then the team need to prioritise the user stories in that release based on a different set of criteria, such as:

What are the risks around feasibility of delivering this story
What else is dependent on this story
How likely is this to break something else
How long do we need to test this story

The Product Manager probably isn’t the best person to make these more granular prioritisation calls. Engineering are more qualified to understand a lot of the criteria above, based on believability-weighted decision-making.

Better Collaboration

Lots of tips on better collaboration – less talking, more intuition, stricter time-boxing.

I’ve often tried to include the whole team (maybe 8-10 people) in decision-making, but Jeff Patton made a good case for running sessions with fewer people. He talked about teams having a mix of deciders and executors, and having a core trio of Product Manager, Design Lead, and Engineering Lead, to lead on making a lot of decisions.

The course was technically a Certified Scrum Product Owner course, but the time we spent on Scrum itself was mainly focused on ways in which we can adapt it.

The things listed above are those sticking in my mind right now, but there was heaps of other good stuff over the two days, and loads to apply right away.

I had to leave early, so thank you, Jeffs.

What’s in a name – Owner, Manager, or Leader?

My esteemed colleague @benjiportwin just wrote a parting post which talks about job titles, and how much they matter, if at all.

He opened with the the Product Owner vs. Product Manager job title thing, which I’ve also been thinking about.

When I joined the NHS Choices team a few years back we had Product Leads who each looked after a specific area of the service. They did a great job of defining the changes needed for their particular products, but didn’t always interact directly on a day-to-day basis with the people building those products.

Changing titles to indicate change

We spent a couple of years changing this as we implemented agile methods across the programme. At the time I pushed for these roles to be called Product Owners, mainly because I wanted to force a distinction between the old and the new way, and that’s what the methodologies we were adopting (like Scrum) tended to call that role.

Shared ownership rules

I tend to associate the Product Owner role title with Scrum, and over time have gone off it a bit. Partly because I don’t like the idea of sticking with just one fixed methodology, and partly because it could imply one person having sole ownership of the product. I much prefer the idea of a team collectively owning the product that they build and run together.

Industry-standard

Instead I shifted towards the Product Manager job title. This seems to be much more of an industry-standard these days. If I see a Product Owner job ad I think “they do Scrum”, when I see Product Manager I think “they have Product teams”. Generalisations I know, but that’s what it conjures up in my mind.

Full circle

Most recently I’ve come back around to Product Lead. I like the idea of somebody leading the development of a product, rather than managing it. I think we all know the difference between a manager and a leader.

Managing a product could perhaps be read as holding it back, pruning it, keeping it in check (thanks to @st3v3nhunt for this). Whereas leading it talks of setting a vision, inspiring progress, and taking the product forward to exciting new places!

Does it matter?

I’ve thought about this mainly because I’ve been taking on a product role myself, but really, as Benji said in his post;

job titles are interchangeable and frankly unimportant, but what matters is the impact you make each and every day.

Good luck in NYC, Benji. See you on the sun deck!

Building consensus around product ideas

I’ve spent the last year or so working with teams in the early stages of Product Discovery – examining the needs of users, and forming ideas and prototypes of services we can deliver to meet those needs.

One thing I’ve seen is the tendency in those teams to have the same conversations about a particular topic or idea several times over. Sometimes in the office, sometimes over lunch, and sometimes in the pub after work (which is where a lot of good ideas form).

This isn’t necessarily a bad thing while we’re shaping our ideas and building consensus in small groups. But once we’ve found ourselves repeating, and agreeing, the same concepts a few times, it feels like it’s time to get things out of our heads and onto some paper, so that we can share the idea further, and build a broader consensus.

The worst case scenario is if we don’t do this, that a small group forms an idea and assumes everyone else is also thinking the same thing. In the early stages of delivering a new service for users, building shared understanding across the whole team is critical.

Audio recording

A nice idea we tried recently was to get a small group of us (three worked nicely) to record the conversation. Kudos to @evilstreak for taking inspiration from the work @mattsheret has done with us on blogging.

We found a quiet space and used a USB mic (that we use for remote meetings) and Quicktime to record our conversation for about 30 minutes.

We didn’t write anything down beforehand, but the conversation was fairly structured, as we’d already had similar versions of it before.

One side-effect of the recording was that we were a bit more conscious of what we were saying, and avoided any unnecessary rambling. Being aware of the recording also stopped us from talking over each other at all. This is something we sometimes suffer from, as everyone has so many ideas to share, people just want to get them out there.

After the recording we transcribed it verbatim into a document. By sharing the audio file, two people were able to work in parallel for 30 minutes to get the whole conversation transcribed.

Pro-tip – using VLC to play back the conversation at a slightly slower speed (about 80%) allowed the typist to keep pace with the conversation without having to pause and replay bits. If you’re a really fast typist maybe you won’t need this.

Listening again to the recording actually helped ideas to crystallise in peoples’ minds. You were listening more intently, rather than thinking about what to say next, as you might in conversation.

Getting feedback from the wider team

As it was just three of us that had the initial conversation, we needed to gather feedback from the wider team.

Initially sharing the transcript was a useful way to share the ideas. Teammates reported that reading the transcript was a very natural way to read, and understand the ideas.

We then began to structure it into a more coherent narrative. Using a tool that enables quick electronic collaboration (like Google Docs) is really handy here. Everyone can be in the same version of the document at the same time – teammates can add comments, responses can be seen instantly, and everyone can see updates as they’re being made.

Crits

We supplement this electronic collaboration with group crit sessions.

Anyone interested in providing feedback gathers around a table and we read through some ideas and have a time-boxed discussion with the goal of gathering specific change to make to the narrative.

In this case we quickly realised that we didn’t have consensus across our team. Lot of new ideas came from these sessions, and the original narrative gave us a way of framing this discussion.

A picture speaks quite a few words

Another (fairly obvious) way of clarifying thoughts and ideas was to do quick sketches of your thoughts. Drawing something and showing it around was a really quick way of prompting further conversation to build shared understanding.

It’s handy to draw something and then pass it to a colleague without explaining what it shows, then ask them to explain their understanding of it.

Building consensus in public

So we’ve built some consensus amongst ourselves, within our team. Next we need to build some consensus with the wider world outside of our team – with our stakeholders and with the public.

We need to be open with our ideas as early as possible for a couple of reasons –

All the services we’re delivering are created for, and paid for by, the public. We should be transparent about how we’re spending public money.
The health and care system is complex. There are many different organisations involved. Being open about our ideas and proposals early on is a good way of getting the message out to these organisations, and starting a conversation with them.

One way in which we share our ideas openly is by blogging.

The document and drawing described above were drafted into an initial blog post by @evilstreak and @paul_furley, the drawing was redrawn by @demotive, and the resulting post is here – Booking is critical for transforming healthcare

My first User Research

Note – I originally posted this here on the NHS Choices blog back in February. It was written as three separate posts about User Research, from the point of view of someone who hasn’t been involved in this kind of thing before.

Over the last fortnight I’ve been observing User Research with my team, and it has been quite an eye-opener.

We’re at the point in our Discovery phase where we’ve made a bunch of assumptions about our users and their needs, and gathered information around these assumptions from various sources – on and off-site analytics, existing literature and research, social media, our service-desk tickets, and on-site surveys.

Now it’s time for us to talk directly to some USERS*

* Not all of the people we interview are necessarily users of the current NHS Choices service. Some of them might be potential users too.

User Research like this isn’t new to us. NHS Choices has had a dedicated Research team since 2007, but it’s in the last year or so that we’ve really started to more tightly integrate the work that the researchers do, into our delivery cycle. This is the first time we’ve involved the whole of the multi-disciplinary transformation team in observing and note-taking for the research sessions, doing the analysis and deciding on next steps within a couple of intensive research days.

Who do we interview?

For the two topics we’re focusing on right now – we’ve been talking to two distinct groups of people

Parents of children who’ve had Chickenpox in the last three months
People who sought a new Dentist in the last three months

We make sure we talk to a mixture of men and women from different socio-economic groups, of different ages, and with differing levels of internet skill.

We ask some quite detailed questions, so it’s good to get people who have had a relatively recent experience (hence the three month time-window) as the experiences they’re recalling will tend to be more accurate.

We use some dedicated participant recruitment agencies to source the specific people we want to interview. We supply a spec, like the parents described above, and they go and find a selection of those people. Obviously there’s a cost attached to this service, but the recruitment can be time-consuming, and it would be difficult to find a big enough cross-section of people ourselves. Outsourcing this to an agency frees up our researchers to focus on the actual research itself.

The setup

We do some interviews in the participants’ own homes – interviewing people in their own environment gives us a much better sense of how people look for information and where this fits into their lives. Also we get to meet participants who would not want to go to a viewing facility.

We also do interviews in a dedicated research facility – these are the ones that the rest of the team and I have been observing.

We’ve used a couple of facilities so far, one in London, and SimpleUsability in Leeds – just a five minute walk from our Bridgewater Place office.

Our interviews have been one hour long. The participants sit with a researcher – who conducts the interview – and a note-taker in the interview room. The note-taker might be another researcher or other member of the team – we’ve had UX Architects and service desk analysts taking notes in our sessions.

With the participants, the researcher and the note-taker in the interview room, the rest of the team are behind a one-way mirror with the sound piped in, observing the whole show.

And yes, with the one-way mirror, it fell to @seashaped and myself to make all the obligatory unfunny gags about being in a police interrogation scenario…

Interviewing

The interviews are based around a Topic Guide prepared beforehand by the researcher. This is based on input from our previous research, and includes specific subjects around which we want to learn more. The whole team feeds their ideas into the Topic Guide.

The interviews aren’t run strictly to the guide though – we’re talking to people about their lives, and the health of them and their families, so naturally the discussion can wander a little. But our researchers are great at steering the discussion such that we cover everything we need to in the interviews.

We decided not to put any prototypes in front of users in the first round of research. We’re trying to learn about users’ needs and their state of mind as they’re trying to fulfil those needs, so we didn’t want to bias them in any way by putting pre-formed ideas in front of them.

We did run a card-sorting exercise with users in the first round of Dental research – getting the participants to prioritise what would be most important to them when searching for a new dentist, by letting them sort cards.

We had a camera set up for the card-sorting exercise, so we could all see it clearly, without crowding around the mirror in the observation room.

As the interview takes place, the note-taker is busy capturing all of the insights and information that come up. As the participant talks, the note-taker captures each individual piece of information or insight on a separate post-it note. This results in a lot of post-its – typically we’ve been getting through a standard pack of post-its per interview.

GDS have written in more detail about some note taking good practices.

Sorting into themes

Once the interviews are over, we have to make sense of everything the users have told us. We have a whole load of insights – each one logged on an individual post-it note. We need to get from what the users have said, to some actionable themes, as quickly as possible, without producing heavyweight research reports. We use the affinity-sorting technique to help us do this.

This basically involves us sorting all of the post-its into themes. We’ve been having a stab at identifying themes first, and then sorting the post-its into those groups first. As the sort takes place we’ll typically find that a theme needs to be split into one or more themes, or sometimes that a couple of existing themes are actually the same thing.

This isn’t the job of just the researcher and note taker who conducted the interviews. The whole team that’s been carrying out and observing the User Research takes part in this process, shifting post-its around on the wall until we feel we have some sensible groupings that represent the main themes that have come out of the interviews.

Dental Affinity Sorting

Chickenpox Affinity Sorting

Although we’re not presenting our research findings as big research reports or presentations, we are logging every insight electronically. After the sort, every insight gets logged in a spreadsheet with a code to represent the participant, the date and the theme under which the insight was grouped. We’re reviewing our approach to this, but the idea is that over time this forms our evidence base, and is a useful resource for looking back over past research, to find new insights.

Hypotheses

Once we have our themes we have to prioritise them and decide what to do with them next. At this early stage this usually means doing some more learning around some of the important themes. We’ve been forming Hypotheses from our Themes – I think this helps to highlight the fact that we’re at a learning stage, and we don’t know too much for sure, just yet.

We’ve been playing around with the format of these Hypotheses. As an example, one of the strong themes from our first round of User Research on Chickenpox was around visual identification. We expressed this as follows –

We believe that providing an improved method of visual identification of Chickenpox

for parents

will achieve an easier way for parents to successfully validate that their child has Chickenpox.

When testing this by showing a variety of visual and textual methods of identification to parents of children who’ve had chickenpox

we learned …

So we will…

If you’re familiar with User Stories, you can see how this hypothesis would translate into that kind of format too. You could argue that all User Stories are Hypotheses really, until they’re built and tested in the wild.

Low-fidelity Prototypes

In order to test this hypothesis, we’re going to need some form of prototype to put in front of users. We’re working on a weekly cycle at present so we only have a few days before the next round of research. Speed of learning is more important at this stage, than how nice our prototypes look, so we’re just producing really low-fidelity prototypes and presenting them on paper.

For the visual identification hypothesis, here are some of the prototypes we’re presenting to users in our second round of research – see what I mean by low-fidelity, but this is just what we needed in order to explore the concepts a bit further, and learn a bit more.

We’ll base some of the questioning in our second round of research around these prototypes, and capture what we learn in our hypothesis template.

Based on this learning from the second round of research, we’ll either capture some user needs, write some new hypotheses to test, create some further prototypes to test, or maybe a mixture of all three.

Side effects

One interesting side-effect of our research sessions that we noticed was that some users were unaware of some aspects of our existing service, and as @kev_c_murray pointed out, some users left the sessions with an increased knowledge of what is available to them.

With comments like “Yeah I’m definitely going to go and look that up on your site now.” – we’re actually driving a little bit of behaviour change through the research itself. Okay, so if this was our behaviour-change strategy we’d have to do another 7 million days of research to reach the whole UK adult population, but every little helps, right…

What have we learned about how we do User Research?

The one week cycle of doing two full days of Research, then sorting and prototyping, is hard work. In fact it probably isn’t sustainable in the way we’re doing it right now, and we’ll need to adapt as we move into an Alpha phase.
Do a proper sound check at the start of the day – in both facilities we’ve used we’ve had to adjust the mic configuration during or after the first interview.
Research facilities do good lunches.
The observers should make their own notes around specific insights and themes, but don’t have everyone duplicating the notes that the note-taker makes – you’ll just end up with an unmanageable mountain of post-its.

More please

Lean UX Book We plan to do much more of this as we continue to transform the NHS Choices service. As we move into an Alpha phase, we’ll continue to test what we’re building with users on a regular basis – we’ll probably switch from a one-week cycle to testing every fortnight.

As someone from more of a Software Development background, I find it fascinating to be able to get even closer to users than I have before, and start to really understand the context and needs of those people who we’ll be building the service for.

If you’re interested in reading more around some of the ideas in this post, try Lean UX – it’s a quick read, and talks in more detail about integrating User Research into an agile delivery cycle.

But why no technical stories?

In my current workplace we’ve been using User Stories in various guises for a while now. One of the things that frequently crops up is whether these Stories can or should be technical or not?

To start with it may be useful to remind ourselves of some of the aspects of a user story…

A User Story describes something that a user needs to do, and the reason they need to do this. They are always written from a user’s point of view, so that they represent some value to the user, rather than a technical deliverable.

They represent the who, what and why – an example might be –

As an expectant parent
I want to receive emails about parenting
So that I can read information about how to best care for my child

They are intentionally brief, so as to encourage further conversation, during which the needs of the user can be explored further, and potential solutions discussed. They are not intended to be detailed up-front specifications – the detail comes out of the conversations.

So who owns the User Stories?

User stories are designed to represent a user need. Most of the time these users are members of the public, but we don’t have our actual users in the office writing and prioritising stories, so we have a proxy for them instead – which we call the Product Lead (PL).

Part of the PL’s job is to represent what our users need – they use User Stories to capture these needs, ready for future discussion. So the PL owns the stories and their relative priority. If this is the case, then the PL needs to understand the stories, so that they can own them. If the backlog has technical stories in it, then it is difficult for them to prioritise these against other user needs.

For example, if the PL sees a story about enabling the public to search for GPs, and another story about reconfiguring a data access layer – they’re likely to prioritise the user-focussed story as they can see the tangible benefit. The technical work is totally valid, but it needs to be derived from a User Story – everything should start with a user need.

What if it’s an internal user?

As suggested above – most of the time the users we are capturing the needs of, are members of the public. Sometimes we use more specific personas in order to capture the needs of specific groups of people e.g. expectant parents

Other times, the users are our own internal users. We can still express their needs in terms of a user story –

As a data manager in the Data Workstream
I want to configure search results views
So that I can change the information displayed in accordance with DH policy

Here there is still an underlying need of the public, but the story is expressed from the point of view of the Data Manager. We could have just written down a technical story like –

Build a data configuration system for results views

but if we do this we have skipped a step. We have assumed that we know the single best solution straight away. Maybe there are other options to achieve their initial need – maybe if we stop to think about these other options we will find one that is cheaper/better/faster too.

When should we do technical design?

Although we need to some technical design up-front in order to set a general direction of travel, the detailed design work for a particular story should be done as close to actually implementing the story as possible.

In the past we have suffered from doing lots of detailed design of technical implementation well in advance of actually being ready to deliver that piece of work. Often by the time it came to deliver that piece of work our understanding had evolved and the up-front design was no longer valid.

Don’t try to do the technical design work when you first create the story. Wait until we are ready to deliver that story, and then look at the technical options available. By doing this work Just-In-Time we are much less likely to waste effort thinking about a solution that will never be delivered.

How do we track progress?

We track progress in terms of completed stories. If we keep those as User Stories then we are measuring our progress in terms of actual value delivered to our users.

If we break stories down into lots of technical stories then it may look like we are making lots of progress, and that we are very busy, but we could have delivered very little genuine value at the end of it all. If, for example we reported that –

“We’ve completed 90% of the business layer code”

that sounds very positive, but we could have delivered no actual working tested functionality for our users at this point. By keeping our stories user-focussed, our progress is also measured in terms of value delivered to users.

How do we get from User stories to technical scope

We’ve talked about how important it is to start with user needs, but ultimately we need to build something, so we have to get down to the technical detail at some point.

One way of ensuring that all scope maps back to an overall goal is to use a technique called Impact Mapping. We used this on a very technical project to ensure that all of the technical deliverables mapped back to an overall goal.

At the same time as deriving those initial stories, we’d usually be thinking about the overall technical approach. We’d look for answers to high-level questions around what technologies to use, and what approach to use. These wouldn’t be technical stories though – this would likely be documented in a lightweight high-level design document.

Story Decomposition and Technical Tasks

Once we’ve derived some initial user stories from our goal, we’d continue to break those stories down until we arrive at the technical scope.

User stories can be split into smaller stories but we always try to retain value to the user in each story, rather than making them technical.

For example, the story above about parenting emails might be split into smaller stories like –

As an expectant parent
I want to sign up for email notifications
So that I receive useful information about caring for my baby

As an expectant parent
I want my email address to be validated
So that it is clear when I have entered an invalid email address

As an expectant parent
I want to provide my first name when signing up
So that the emails I receive are personally addressed to me

Each of these stories is a smaller deliverable, but still makes sense from a user’s point of view.

Further to that, once we end up with nice small stories, we can create a list of technical tasks. Each story might contain the tasks needed to deliver that particular story. The tasks get down to the level of technical detail around what components and packages need to be altered, in order to deliver.

Ultimately – we will end up with technical pieces of work to do. Key is that all of these are derived from user needs.

* We don’t have to use User Stories for EVERYTHING

Okay, so we go on about User Stories a lot, but ultimately they’re just a tool for communication. A User Story represents some change we want to make in order to deliver some value. It’s a cue to go and have a further conversation about that piece of work, and that value.

If we can have these conversations, and deliver the value, without writing stories every time, then maybe that’s okay. The most important thing is for the people concerned to talk to each other frequently, deliver the work in really small chunks, and get feedback as often as possible.

User stories do really help us with this, but there might be occasions when they’re not the best tool…

Conclusions

Yes, we’re going to end up with stories that are technical in their implementation, but it’s important to not jump straight into that implementation. Think about our users, and their behaviours – and derive the stories from that.

Friends & Family Tweet

For my 10% time project I have decided to look at whether Friends and Family Ratings could be submitted via Twitter.

The idea was that if you’re sat in a GP waiting room you might be tweeting, and if you were shown a particular hashtag(s) then you could use these to tweet a view on the service you’re receiving.

This example could be flawed in several ways, as you’re more likely to want to give an opinion after you’ve actually seen the doctor, not while you’re waiting there…

However I’m going to explore the possibilities and see where it takes us – there may be mileage in integrating with Care Connect so that people can submit real-time ratings, comments or complaints via Twitter. It could be extended beyond Twitter itself to other channels. We could make use of geo-location (thanks Russell) or QR codes (thanks Sean) to enhance the experience.

EDIT – there seems to be appetite for this today
http://www.bbc.co.uk/news/health-24669382

The technical bit

So far I have a basic Gherkin spec which looks like this –

Feature: Grab Tweets By Hashtag
In order to store tweets for processing
Twit can grab tweets with a specified hashtag from twitter

Scenario: Grab a new tweet with a hashtag
Given there is a new tweet with the hashtag #nhsfft
When you wait 15 seconds
And you run Twit
Then this tweet should be stored

In the background this automatically posts a tweet from a personal dev account I set up, and then runs a console app (called Twit, right now) which searches twitter for tweets with this hashtag and then stores the tweets in a RavenDB store for future processing. The 15 second pause is there as the tweet wasn’t showing up in search results immediately – this sucks a bit as it slows down the automated test…

The next step is to write a separate process, which takes these stored tweets and posts them to Choices via our comment capture API.

I’m trying to practice what I preach (!) and use a BDD and TDD approach to the coding, with lots of refactoring as I go to ensure the code-base stays simple and clean.

It’s also good for learning – I found that you don’t have to store the NuGet packages in source control (thanks Steve), and am learning bits and pieces about RavenDB too.

Interacting with Twitter poses some challenges too. I began writing raw requests, but found dealing with OAuth was eating up time (when I just want to get something working) so am now using a couple of .Net components that wrap around the Twitter API. It’s likely that I’ll have to move from using the REST API to the Streaming API too, in order to ensure we don’t miss any tweets.

I’m also learning more about git – it’s only in a local repository at present, but will hopefully be in github soon!

10% time

We just launched a new initiative in our team to encourage more innovation and learning. Our 10% time is a bit of a cross between Google’s 20% time, and Dragons’ Den.

The idea is that any team members in Product Delivery can spend Friday afternoons working on their own project. It can be anything they want, but our panel of Dragons must agree to ‘invest’ a certain number of afternoons in the idea first.

Pitch

To get involved – team members deliver a short pitch explaining

what the idea is
how it will benefit the wider organisation
how long they initially need to work on it

Investment

The Dragons can choose to invest a certain number of afternoons in the idea, or ask for more refinements to the idea. When the end of the invested time comes – we’ll have demos of the different ideas, to see what we’ve come up with.

Ideas

There are some already some interesting ideas being pitched – ranging from a historic twitter analysis and a pregnancy care planner app, to a virtual fridge. Some people have pitched individually, and others have worked in pairs. The methodologies, technologies and tools used to build these ideas are entirely up the individuals building them, so it’s a chance to try out some new stuff.

What does a Product Lead do?

The Product Lead (PL) role has existed in our organisation for some time. On the face of it, the role is broadly comparable to the Product Owner (PO) role, as described by the Scrum agile methodology – some of the responsibilities match up.

The main difference is that the PO role is generally regarded as a full-time role working very closely with the delivery team, whereas our PLs are almost always individuals who have other full-time roles within their respective workstreams.

This poses problems, and creates frustration, as we have competing priorities for our PLs’ time. We could try to enforce the traditional PO role, but really it is for us as an organisation to define our roles, and what they do.

In an attempt to clarify the role, we wrote this guide for our Product Leads, around the things that we expect them to do –

Vision and Goal

You’ll be outlining the vision for the product or feature, and explaining the overall goal. Ideally this goal would be measurable. If you’re not sure what the overall goal is, try asking yourself ‘Why?’ until you reach the real value – see The 5 Whys

Detail

You’ll work with the delivery team to help define the features we’re going to build. We do this through User Stories and their Acceptance Criteria. The best way to do this is to provide us with specific examples – lots of them.
e.g. How should it behave when we do this? What should users be able to do in this scenario?
We have BAs to help with this, but you need to own the backlog of User Stories that will be built.

Value and ROI

You’ll need to justify your product, in order to obtain a project budget. Once we start working on your product or feature, you’re responsible for guiding the team to build the right thing, that delivers value and meets the goal you’ve defined. Any information you can provide on what impact or benefit the product has had once it’s been released is useful too.

Prioritise

As we’re working within a limited project budget, its important that we build things in the right order. We don’t want to use the budget up working on non-critical features. The best way to present priorities is a numbered list in priority order. You own the prioritisation, and you can decide what we should build next, as we learn more about what our users need.

Verification

We need you to get involved in verifying that we’ve built the right thing, as early as possible. The testing doesn’t all wait until the end any more. We can provide you with a link so that you can see your product growing in a test environment on a daily basis. The more closely involved you are with verifying the product, the better you will be able to prioritise future work, and shape the product.

Feedback

We love feedback! Especially from real users. It’s what enables us to build better products. The weekly review is a great opportunity to provide feedback on what is being built.

Meetings

We have a number of meetings during the development of your product. First of all you’ll need to work regularly with BAs, Architects and UX Designers in coming up with some early ideas and design work. Once the development and testing work kicks off we have daily stand-ups, weekly Reviews and Planning sessions, and Retrospectives where we look at how things are going. You need to be involved in as much of this as possible.

Team

Whilst your product is being designed and built, we’d like you to be part of the Delivery Team. We know you all have busy jobs to do, but the best deliveries happen when the Product Lead gets closely involved.

Proxy

You must represent other stakeholders in the business, including your Managers. Make sure you manage their expectations, and let them know about any decisions that are being made, as early as possible. You are the main line of communication between stakeholders and the team delivering the product.

Documentation

You’ll need to update the documentation for your product in our Product Wikis. This involves creating and updating wiki pages so that the relevant people across the organisation understand, and can support, your new product or feature.

Impact Mapping on ODS

Impact Mapping is a technique for deriving scope from goals. It’s useful for looking at different options for meeting a particular measurable goal.

I’d been keen to try impact-mapping-cover it out since picking up a copy of Gojko’s book on the subject at BDDX2012. I don’t think we’ve used it ‘end-to-end’ yet – to truly evaluate and measure different options – but I have found it really useful in deriving scope and ensuring that scope maps back to an original goal.

A good example of where we used Impact Mapping to derive User Stories on a very technical project was the ODS migration project. Due to the restructure of the NHS – removal of SHAs and PCTs, and introduction of CCGs and ATs – we had to do some work in order to reflect this structure to the public.

This involved extensive changes to multiple ETL packages, and changes to the code to generate organisation profiles. We could easily have dived straight in to the technical detail and decided which ETL packages to change.

Instead we went through an exercise to ensure that all of the technical work we were to undertake was mapped right back to the overall goal.

1. Goal

Firstly we established the overall goal that we’re working towards. In this case the goal is for the nhs.uk site to accurately reflect the new NHS organisation structure.

Normally with this technique we’d want the goal to be measurable – like an increase in profit of 5%, or an increase of 10000 subscribers. In this case the goal is effectively binary – either the site does reflect the new structure, or it doesn’t…

This is the cell at the centre of the map.

* In hindsight maybe we could have been more rigorous with the goal and measured it by surveying site visitors as to their understanding of the new structures. This may have been a better measure of the true overall goal – which is to communicate the new structure.

2. Actors

Next we went through an exercise of mapping out the different users, or actors associated with the goal of reflecting the new NHS structure, for example –

The external organisations who provide data feeds into nhs.uk – ODS, PPD, EDOS, PCIS
Organisation profile content managers
The general public themselves, as they are going to view the new organisation profiles on the site.

The actors are represented by the pink cells surrounding the goal. ‘Big Shots’ is a role that my colleague Ashish came up with – the senior stakeholders who use the accountability views in our ‘Find & Compare’ product.

In future I’d like to identify more specific roles, and possibly tie them in with the personas that our UX team use.

3. Impacts

After identifying these different roles, we looked at the different ways in which they contribute to reflecting the new NHS structure, or how their behaviour will change as a result of the new structures. What are the impacts we want to have on the actors, or the impacts that we need the actors to make.

The blue cells represent the impact. The map really starts to expand at this point as some roles can do many things to bring us towards our goal.

4. Scope

Once we’d identified all of these behaviours we could then start to derive the scope required to enable that behaviour to occur.

So for example – ODS provide a nightly feed of the new NHS organisations, this is how they contribute to our goal of reflecting the new NHS structure. The work we need to do to enable that is to create a series of nightly import and data-synchronisation processes.

These can be derived as a series of user stories – something like

In order that nhs.uk can reflect the new NHS organisational structure
ODS can provide a nightly feed of CCG data

Each story is represented by a green cell. The overall map looks like this –

Some of these stories were high-level and were then broken down further – we just added another level of nesting of green cells to represent this break-down.

Visualising the backlog

I’ve found the map a really useful alternative way of presenting the product backlog for a particular project or piece of work.

Since we started using User Stories to represent requirements, some have commented that in doing so we lose sight of the bigger picture.

One thing a colleague mentioned the other day is that User Stories are like leaves on a tree, but when we put them into the backlog it’s like stuffing all the leaves into a bin bag – making it pretty difficult to maintain any context between them.

Now with our Impact Map we’ve re-formed the tree with the leaves in place. We can stick our map up on the wall, and cross items off as they’re completed, or make notes against them.

Living Document

We had a good stab at capturing the necessary scope up-front, but naturally things emerged as we progressed with the work. Having this Impact Map meant that we could always check where suggested scope fitted into the big picture – how did it help us get towards our goal.

“Does it help us meet the goal?”

Some technical work was suggested around changing the mechanism by which we import certain data feeds, but when we examined it in the context of the map, it turned out we didn’t really need to do it. If we had jumped straight into the technical work without looking at the users, their needs and impacts, we would have wasted time doing this unnecessary work.

Tracking

I was acting as BA on this project, and I personally found the map a really useful way of tracking scope, progress towards our goal, and what our priorities were. The map itself was useful during planning, prioritisation and design sessions, as we had a visual representation of the scope in front of us.

Each green ‘scope’ cell was set up to link to our work tracking system, and it was easy to visually represent progress on the map by ticking off or shading the completed stories. This showed how much closer we were getting to our overall goal.

The map was also used to show up blockers and dependencies in scope. We used icons in Freemind to indicate key questions and blockers that arose during delivery.

Tools

For the map described in this post I used FreeMind. Once you have a few keyboard short-cuts set up it’s quick to colour-code the cells and add icons and hyperlinks in.

I also tried out a MindMup version too.

Next steps…

The big thing I felt was missing from this first attempt at Impact Mapping was making the goal measurable. When I first created the map I didn’t think there was much in the way of metrics that we could use to measure our progress towards the goal – there’s certainly not a monetary value from our point of view. We could look at measuring the amount of traffic that certain areas of the site receive before and after implementation, and we could measure the number of service desk calls we receive in relation to the NHS structural changes.

This is something we need to work on – tying the features we deliver back to explicit, measurable goals. I think it really helps the whole delivery team understand what we’re aiming for, and if we can keep our options open for meeting that goal, and measure our progress towards it, then all the better.

We’ll keep refining our use of this technique on further product developments. We’ll push to get the Impact Map created as early on as possible, when we’re still figuring out the overall goal.