Joe Blogs

Working as a Business Analyst

I spent much of the last year doing contract work as a Business Analyst in a digital delivery team. I wanted to note down some of the ways in which I’ve approached this role.

It’s not a role I’ve formally held before, but I’ve been able to apply my experience of working in roles like Technical Lead, Delivery Manager and Product Manager.

Favour storytelling over story writing

To deliver the right product it’s important that we build shared understanding about our users, their needs, and how we’ll go about meeting those needs.

User stories help us with this. Everyone likes a good story.

Going back to the original idea of user stories; we need to tell stories, and have conversations, to build that shared understanding. Not just write them down.

To quote Jeff Patton:

“Stories aren’t a written form of requirements; telling stories through collaboration with words and pictures is a mechanism that builds shared understanding.”

The initial user story card is a starting point. I try to write just enough to be able to tell a story, and begin those conversations.

Use fewer words

I try to apply what I’ve learnt from Content designers I’ve worked with. We put a lot of effort into creating clear content in the services we deliver. Why not do the same when communicating in our teams?

The general approach can be summarised by this classic Orwell quote:

“Never use a metaphor, simile, or other figure of speech which you are used to seeing in print. Never use a long word where a short one will do. If it is possible to cut a word out always cut it out. Never use the passive voice where you can use the active. Never use a foreign phrase a scientific word or a jargon word if you can think of an everyday English equivalent. Break any of these rules sooner than say anything outright barbarous.”

A picture speaks a thousand words

One way to use fewer words might be to draw a picture instead.

A picture on a screen or whiteboard is often clearer than a load of words.

Drawing pictures together as a group helps to explain ideas more clearly than showing a picture that was drawn earlier.

Some pictures are transient, and only needed for a specific conversation. Pictures that we want to refer to later are photographed and uploaded to Slack or Confluence.

Explain abbreviations

I work in an environment that is littered with obscure abbreviations. Explaining them when they’re first used in any documentation is helpful, particularly for people new to the domain.

Publish don’t send

I avoid sending documentation. Instead I publish it in one place, and share the link. Then everyone knows they’re looking at the latest version.

Tools like GitHub and Confluence mean the days of emailing documents around should be over.

If someone can’t access a particular link (network issues, anyone?), I export the content and send as a PDF, to make it clearer that this is a snapshot.

Elaborate little and often

It can take time to digest complex technical and domain concepts. Running lots of short sessions to explain things means that people have time to go away and think, and then come back and discuss them. It’s how we learn new things.

I usually bring one or two new features to a weekly elaboration session lasting under an hour. Sometimes we review the same feature in 2 or 3 sessions, as we learn more about what’s needed, and what our approach might be.

I consult with my teammates on the best time for scheduling these sessions, to avoid disrupting their Maker’s schedule.

Be the glue

I work in a multi-disciplinary team of Researchers, Designers, Developers, and Testers. Working as a Business Analyst means I can help bring these people together.

Faciliatating sessions like sketching workshops helps the team to come together to solve problems. It binds the team.

Take responsibility for product decisions

I built a trusting relationship with our Product Manager. This helped me to judge when to take decisions about the product myself, and when to consult others.

If I was doing this on a new team, I might try something like Delegation Poker, to agree who can make different levels of decision about the product.

By taking responsibility at the right level, I worked as a proxy product owner for an operational service we delivered.

What else?

These are some of the ways in which I’ve approached the Business Analyst role.

Are you a Business Analyst, or do you work with one?

How else do Business Analysts approach their work?

Distraction-free phone

I reconfigured my phone to reduce distractions. Now I spend less time on my phone, and more time doing things that are important to me.

The problem I was trying to solve

Checking certain apps on my phone had become a default action for me. I’d feel an urge to check them many times a day. Email, Slack, Twitter and The Guardian are my guilty pleasures.

Sometimes a quick check could turn into 20 minutes of browsing and reading content. At the same time, I was struggling to find time to do other things that are more important to me.

Make Time

A few months ago I read Make Time by Jake Knapp and John Zeratsky (the guys who wrote Sprint).

Make Time describes the apps that suck up our time through endless streams of content as Infinity Pools. They’re designed to get you into a dopamine-induced loop that keeps you checking for updates,

For some of us willpower isn’t enough to stop going back to them. You have to break the loop by changing your personal defaults.

The book describes 87 tactics for prioritising things that matter to you and staying focused on them, by reducing distractions and increasing your energy levels.

I’ve been trying some of these tactics, including #17 Try a Distraction-Free Phone.

Reconfiguring my phone

I did the following to reconfigure my phone:

1. Remove social media apps

This was easy. I’ve been gradually removing these anyway, and only had Twitter on my phone. I’ve removed that and re-added it a few times in the past.

This time I’ve terminated Twitter for good.

2. Remove news apps

I have a love/hate relationship with news.

I’m a sucker for US political news right now, in the age of Trump. But I’m not convinced how healthy it is to know what’s going on thousands of miles away, all of the time.

The Guardian app is now gone.

3. Remove email and Slack

I spend enough time in these when I’m on my laptop. I don’t need them on my phone.

4. Remove browsers

I removed Firefox, and then disabled Safari via the ‘Screentime – Allowed apps’ settings.

It feels a bit weird that I can’t click on links that people send me via Whatsapp. But if I really need to look at something, it’s easy to re-enable Safari.

You might ask what the point of having a smartphone is, given I’ve just removed these apps from it. But I still use my phone for loads of useful things, like:

getting directions
buying train tickets
listening to music
moving money
taking and uploading photos

I’ve just removed some of the things that were distracting me.

Combining with other tactics

As well as removing these Infinity Pools from my phone, I also tidied it up. Tactic #20 Clear Your Homescreen, left my homescreen looking like this.

This is nice and clean. It also means that if I open my phone to check a message, I’m less likely to get drawn into looking at other apps.

Tactic #19: Nix Notifications left only those 3 apps on the homescreen with any kind of notification enabled. I have to manually check all of my other apps.

One of the main reasons that I would pick up my phone (and then sometimes be distracted by it) was to check the time. The battery died on my watch and I never got around to replacing it.

So I tried Tactic #21: Wear a Wristwatch. I bought a new watch, and now I check my phone less. Simple.

I removed Twitter and The Guardian from my phone, but I was still checking them pretty regularly on my laptop.

So I also tried Tactic #28: Put a Timer on the Internet. I set up Leechblock on my laptop, to only allow access to those 2 sites between 7am and 5pm. Now I don’t get distracted by news and Twitter in the evenings, or first thing in the morning.

Making time

Through a combination of these tactics, I spend a lot less time distracted by my phone and laptop now.

I have more time to do things I value. I’ve started reading more books. I exercise more.

I’m more aware of my surroundings. If I’m waiting for a bus, for example, instead of pulling my phone out, I can just stand and notice the things going on around me.

I’m more present with other people. When I’m playing with my son, I don’t feel the draw of my phone in my pocket any more. Instead I can focus fully on more pressing matters, like building a “big big big big big Lego T-rex”.

And I’m writing a little more too. Like this blog post. More to come 🙂

Product management takeaways

I’ve just spent two days with two Jeffs (Gothelf and Patton). They ran a great product management course, and I wanted to quickly get down some of my personal takeaway points, while it’s fresh in my mind.

The term MVP has been wrecked

This rang true as one of my bugbears. I’ve always referred to the definition from Eric Ries’ 2011 book The Lean Startup:

The smallest thing you can make or do, to test a hypothesis

But I hadn’t realised the phrase was originally coined in 2001, by Frank Robinson, as:

The smallest product to meet it’s desired market outcomes

I can see why these clash, and how the wording of Ries’ definition causes confusion.

The term has become overloaded where I work, often being used to refer to:

The scope that’s left once we’ve fixed the people working on the team, and the timeframe they have available.

I’m aiming to use the two alternative terms that Jeff Patton suggested, to be more specific in referring to the Robinson or Ries definition, respectively:

Smallest Successful Release

Next Best Test

Defining metrics for impacts and outcomes

This is pretty much what we’ve been doing on our work around self-referral into psychological therapies; figuring out what the key impact is, quantifying that, and then defining measurable outcomes that are leading indicators of us achieving that impact. We’ll do more on visualising this.

Hypotheses are going to hang around

I’ve used hypotheses to track user research experiments and activities, but I think a mistake we’ve made is to think that a hypothesis is done with, either validated or not, after our first experiment.

It’s fine to run a succession of experiments against a given hypothesis, each experiment becoming higher fidelity, as we learn more (see Giff Constable’s Truth Curve).

A hypothesis likely won’t be completely proven until we get something into production at scale. The earlier, lower-fidelity experiments allow us to stop earlier if a hypothesis is disproven.

Visualising multiple backlogs

We’ve tried before to visualise the end-to-end flow of work, from idea through to delivered user stories, and ended up ditching it in favour of a much simpler ‘Todo, Doing, Done’ type of board, for the whole team.

Jeff Patton showed some examples of separate backlogs and boards, for different types of work. I think we should have another go at this, embrace the idea that there are different types of Discovery and Delivery work going on, and visualise accordingly.

This also ties in with the ideas around dual-track development, another thing we could get more rigorous with. There were some great ideas around adapting your ‘scrum’ sessions to play towards these two different types of work going on.

Three ways of prioritising, for three different situations

These different types of work, and separate backlogs, should be managed and prioritised in different ways too:

New opportunities identified should be prioritised against our overall strategy or vision, to ensure we engage with things that keep us heading in that direction.

For Discovery activities we should prioritise based on what we need to learn the most about; assessing hypotheses based on risk and potential value.

Once we, as Product Managers, have defined a minimum successful release, then the team need to prioritise the user stories in that release based on a different set of criteria, such as:

What are the risks around feasibility of delivering this story
What else is dependent on this story
How likely is this to break something else
How long do we need to test this story

The Product Manager probably isn’t the best person to make these more granular prioritisation calls. Engineering are more qualified to understand a lot of the criteria above, based on believability-weighted decision-making.

Better Collaboration

Lots of tips on better collaboration – less talking, more intuition, stricter time-boxing.

I’ve often tried to include the whole team (maybe 8-10 people) in decision-making, but Jeff Patton made a good case for running sessions with fewer people. He talked about teams having a mix of deciders and executors, and having a core trio of Product Manager, Design Lead, and Engineering Lead, to lead on making a lot of decisions.

The course was technically a Certified Scrum Product Owner course, but the time we spent on Scrum itself was mainly focused on ways in which we can adapt it.

The things listed above are those sticking in my mind right now, but there was heaps of other good stuff over the two days, and loads to apply right away.

I had to leave early, so thank you, Jeffs.

Getting remote-friendly

Several teams where I work have been experimenting with different ways to support remote-working.

We’ve done this for a number of reasons:

Open plan offices are not always conducive to concentrating and thinking without interruption. We’ve adapted our workspace as best we can, but it can still be cramped and noisy on busy days. This is fine for the teams running noisy workshop-type activities, but it’s not good for those people who need to get their heads down to concentrate on something for a while.

Related to this; we know that context-switching and interruptions can be annoying, and impact our productivity. Some of the remote-working practices we adopted aim to manage those interruptions.

We thought that doing more remote-working would force us to think more about how we communicate about our work, and make it more open to people outside of the team.

We value flexibility. Our people work hard, and helping them to fit in work around their ‘real lives’ is something we want to support. Working remotely reduces the amount of commuting people have to do, for starters.

Many forward-thinking, and successful organisations are doing this too. We’ve been inspired by the likes of GitLab, Automattic, and 18F.

There are other reasons too. Making these changes also fits with our culture of continuous improvement. We’re always trying to make small adjustments to the way we work, and this seemed like a more ambitious experiment to take on.

Remote-first

The biggest change we’ve made in our team is starting to adopt a remote-first approach.

Remote-first means to communicate and collaborate as if all your teammates are working remotely, even when some of them are in the office.

In practice this means that if any team members are working remotely and join a meeting via Google Hangouts, then everyone joins as if they’re working remotely. So if they’re in the office they join Hangouts individually from their desk, rather than piling into a meeting room.

This was strange to start with, as you had people sat across the office from each other, talking over Hangouts. But then it starts making more sense, as it means that everyone in a given session is on a level playing field. It avoids the situation of having a less good experience if you are working remotely.

It also means that instead of turning up at someone’s desk to ask a question, you might send them a message over Slack instead. Unless you really need the answer right that second, you can communicate asynchronously and let them answer when it suits them to break from what they’re doing. This also means that conversations can be made more ‘visible’ to your other teammates.

This has been, and continues to be, very much a gradual journey of discovery for us as a team. We’ve gradually started spending less time in the office. Some people now come into the office a couple of days a week, and work remotely the rest of the time. A few people still come into the office most days. We’re not a truly remote-first team, but I’d say we’re a lot more remote-friendly.

Some things we’ve learned along the way

It’s really important to ditch the thing you sometimes hear along the lines of “x is working from home, I’ll pick it up with her tomorrow when she’s in the office”. Unless something absolutely has to be discussed in-person, you can probably at least start that conversation right now, over another channel. We still have to pull each other up on this.

We’re more conscious about socialising. When we’re not physically together so often, it’s important to refresh those team bonds that are built through face-to-face interaction. Think less ‘forced fun’, and more just scheduling time in for coffee, team lunches, and team drinks.

Be explicit about whether a meeting is remote or really needs everyone physically together. Over time we’re reducing the number of sessions that are mandatorily face-to-face, but there are still some where we find it helps to be together.

Over-communicate status. When you’re in the office people can see you’re around, and you’re working on certain things. When you’re remote it’s helpful to keep people updated as to how you’re getting on.

Finding the right balance between synchronous and asynchronous communication. Not everything has to be a real-time conversation, but knowing when to switch out of Slack and into a video-call is something we’ve been working on.

When new people join the team, we spend a bit more time in the office together, so we can form new bonds and get used to working together.

Working remotely doesn’t have to mean working at home. Co-working spaces, and the obligatory coffee shop, are regularly used by the team.

People like the flexibility that working in this way gives them. It can help in balancing work, and ‘real life’.

It’s not for everyone; some people prefer coming into the office and being physically co-located with their team-mates. We support that, and in future people’s preferences around remote working should be factored into how we set up new teams.

Challenges

Many of the new practices have been most eagerly adopted by the Developers in our team. There’s a good fit for them with being able to work in a quiet environment, get into the flow of their work, and communicate asynchronously as needed. It helps them find a ‘maker’s schedule’.

We still face challenges as to how we run design processes remotely. We have run decent design reviews remotely; in some ways it forces us to slow down a little and listen better as each person speaks. But there are some activities like analysing user research and sketching sessions, where we still generally choose to be physically in the same place.

Some might say that you cannot be ‘agile’ unless your whole team is all in the same place. I’d challenge that now; I think if you have a well-bonded team, with modern tooling for remote work, you can definitely work in an agile way.

I think on reflection that when we’re in ‘delivery mode’; knowing the product we need to build, and having a clear backlog to work from, then remote-first works well for us. More recently when we’ve been in ‘learning mode’; back in discovery, figuring out what we should be making, then we’ve benefited from being in the office together a bit more.

Is it the future?

A future that’s supportive of us remote-working could lead to us being able to hire people from much further afield, form some fully remote teams, and have much less need for permanent office-space. Some of our recent vacancies have been advertised with remote-working as an option.

It’s not for everyone, but if you’re supportive of remote-working in your teams, do consider adopting practices that fall under the banner of remote-first. If you already do this, I’d love to hear about what you do, and how it works for you.

SNAFU? You have a choice.

You always have a choice.

Is the wifi broken? Again?

You can choose whether to raise it with IT. You can choose to escalate it because it’s causing pain for your team. You can choose to keep doing those things over and over until it gets fixed.

You can choose to engage with the people who can fix the problem. You can choose to understand the problem from their point of view. You can choose to help them to help you.

Or you can choose not to do those things. And instead choose (probably unconsciously) to just moan about it. This is usually an easier choice. But it’s also the worst.

It’s the worst because nothing gets fixed, and because moaning just makes everyone feel bad.

It’s easy to fall into the trap of learned helplessness. To think that nothing can be done, or that only someone else can sort it for you. But that is to feel helpless and completely dependent on others.

Of course, there’s no excuse for non-working wifi in a modern workplace, but if things really are Situation Normal: All F*cked Up, you can choose to start breaking some rules to get things fixed.

You can choose to buy a mobile router so your team has their own wifi. You can choose whether to try claiming this as a legitimate business expense. Or you can choose to all work remotely.

Whether you’re a leader or a follower you always have a choice.

Time to start exercising it.

What’s in a name – Owner, Manager, or Leader?

My esteemed colleague @benjiportwin just wrote a parting post which talks about job titles, and how much they matter, if at all.

He opened with the the Product Owner vs. Product Manager job title thing, which I’ve also been thinking about.

When I joined the NHS Choices team a few years back we had Product Leads who each looked after a specific area of the service. They did a great job of defining the changes needed for their particular products, but didn’t always interact directly on a day-to-day basis with the people building those products.

Changing titles to indicate change

We spent a couple of years changing this as we implemented agile methods across the programme. At the time I pushed for these roles to be called Product Owners, mainly because I wanted to force a distinction between the old and the new way, and that’s what the methodologies we were adopting (like Scrum) tended to call that role.

Shared ownership rules

I tend to associate the Product Owner role title with Scrum, and over time have gone off it a bit. Partly because I don’t like the idea of sticking with just one fixed methodology, and partly because it could imply one person having sole ownership of the product. I much prefer the idea of a team collectively owning the product that they build and run together.

Industry-standard

Instead I shifted towards the Product Manager job title. This seems to be much more of an industry-standard these days. If I see a Product Owner job ad I think “they do Scrum”, when I see Product Manager I think “they have Product teams”. Generalisations I know, but that’s what it conjures up in my mind.

Full circle

Most recently I’ve come back around to Product Lead. I like the idea of somebody leading the development of a product, rather than managing it. I think we all know the difference between a manager and a leader.

Managing a product could perhaps be read as holding it back, pruning it, keeping it in check (thanks to @st3v3nhunt for this). Whereas leading it talks of setting a vision, inspiring progress, and taking the product forward to exciting new places!

Does it matter?

I’ve thought about this mainly because I’ve been taking on a product role myself, but really, as Benji said in his post;

job titles are interchangeable and frankly unimportant, but what matters is the impact you make each and every day.

Good luck in NYC, Benji. See you on the sun deck!

Resources, or People?

Workplace language is interesting. We hear a lot of different jargon and cliches that we wouldn’t ordinarily encounter outside of the office. I think we know some of it is a bit silly and acknowledge it as such, but other workplace language is treated as totally normal.

One of the most common, but perhaps pernicious, examples of workplace language is the use of the word Resources, when we mean People. It really does seem to be workplace-only vernacular too – I’ve yet to hear anyone say “We need some more resources for 5-a-side tonight.”

I understand that it’s a fairly industry-standard term, but that doesn’t mean there aren’t issues with it. Lots has been written on this before – posts like this, and this – and there’s even a day dedicated to the cause.

oxforddictionaries.com defines resources as

A stock or supply of money, materials, staff, and other assets that can be drawn on by a person or organisation in order to function effectively

So technically speaking using the word resources to describe people isn’t wrong, and sometimes it makes sense to use the word resources to describe a particular ‘thing’. Resource management isn’t just about people – it could be about laptops, office space, or beanbags.

But 99% of the times I hear ‘resources’ mentioned at work, it’s being used to refer solely to people.

The problem is that the word resources seems to imply an interchangeable quantity –

“we need more resource”

“add some more resources”

“this resource is leaving, let’s get another one in”

From a management point of view this is great. It would be fantastic to be able to swap interchangeable resources in and out at will, in order to maintain performance.

But of course it doesn’t work like that. People have different skills, personalities, likes, dislikes, attitudes and so on. You cannot simply switch people in and out of a team, and expect things to continue at the same level or pace.

Because in knowledge work, value is generally delivered by teams, not individuals. A team is not just the sum of its parts – it’s the product of its interactions. The relationships between the people in the team determines the success of the team – it’s not about just adding up the raw skills of the individuals in the team.

The word resource obfuscates this fact. It helps us to kid ourselves that we can work in this way – swapping people around. It’s hiding the reality of the situation that when you’re dealing with knowledge work – people are not always going to be completely interchangeable.

That’s not to say I’m not against moving people around between teams on some regular basis. This can keep things fresh, and helps to share knowledge. But I think I’d get better results from trying out a new winger in my regular line-up, than swapping out the entire midfield.

I know a lot of people don’t like being referred to in this way, but I don’t think this is just about individuals being sensitive to being called a ‘resource’ – it’s about our cultural definition of how we view and treat our people, and about how we plan and manage work in a realistic way.

Having shared language is really important for building shared understanding, but next time you’re about to use the the word Resources, maybe pause and think; would the word People explain the situation in a clearer, more helpful, and more realistic way?

Building consensus around product ideas

I’ve spent the last year or so working with teams in the early stages of Product Discovery – examining the needs of users, and forming ideas and prototypes of services we can deliver to meet those needs.

One thing I’ve seen is the tendency in those teams to have the same conversations about a particular topic or idea several times over. Sometimes in the office, sometimes over lunch, and sometimes in the pub after work (which is where a lot of good ideas form).

This isn’t necessarily a bad thing while we’re shaping our ideas and building consensus in small groups. But once we’ve found ourselves repeating, and agreeing, the same concepts a few times, it feels like it’s time to get things out of our heads and onto some paper, so that we can share the idea further, and build a broader consensus.

The worst case scenario is if we don’t do this, that a small group forms an idea and assumes everyone else is also thinking the same thing. In the early stages of delivering a new service for users, building shared understanding across the whole team is critical.

Audio recording

A nice idea we tried recently was to get a small group of us (three worked nicely) to record the conversation. Kudos to @evilstreak for taking inspiration from the work @mattsheret has done with us on blogging.

We found a quiet space and used a USB mic (that we use for remote meetings) and Quicktime to record our conversation for about 30 minutes.

We didn’t write anything down beforehand, but the conversation was fairly structured, as we’d already had similar versions of it before.

One side-effect of the recording was that we were a bit more conscious of what we were saying, and avoided any unnecessary rambling. Being aware of the recording also stopped us from talking over each other at all. This is something we sometimes suffer from, as everyone has so many ideas to share, people just want to get them out there.

After the recording we transcribed it verbatim into a document. By sharing the audio file, two people were able to work in parallel for 30 minutes to get the whole conversation transcribed.

Pro-tip – using VLC to play back the conversation at a slightly slower speed (about 80%) allowed the typist to keep pace with the conversation without having to pause and replay bits. If you’re a really fast typist maybe you won’t need this.

Listening again to the recording actually helped ideas to crystallise in peoples’ minds. You were listening more intently, rather than thinking about what to say next, as you might in conversation.

Getting feedback from the wider team

As it was just three of us that had the initial conversation, we needed to gather feedback from the wider team.

Initially sharing the transcript was a useful way to share the ideas. Teammates reported that reading the transcript was a very natural way to read, and understand the ideas.

We then began to structure it into a more coherent narrative. Using a tool that enables quick electronic collaboration (like Google Docs) is really handy here. Everyone can be in the same version of the document at the same time – teammates can add comments, responses can be seen instantly, and everyone can see updates as they’re being made.

Crits

We supplement this electronic collaboration with group crit sessions.

Anyone interested in providing feedback gathers around a table and we read through some ideas and have a time-boxed discussion with the goal of gathering specific change to make to the narrative.

In this case we quickly realised that we didn’t have consensus across our team. Lot of new ideas came from these sessions, and the original narrative gave us a way of framing this discussion.

A picture speaks quite a few words

Another (fairly obvious) way of clarifying thoughts and ideas was to do quick sketches of your thoughts. Drawing something and showing it around was a really quick way of prompting further conversation to build shared understanding.

It’s handy to draw something and then pass it to a colleague without explaining what it shows, then ask them to explain their understanding of it.

Building consensus in public

So we’ve built some consensus amongst ourselves, within our team. Next we need to build some consensus with the wider world outside of our team – with our stakeholders and with the public.

We need to be open with our ideas as early as possible for a couple of reasons –

All the services we’re delivering are created for, and paid for by, the public. We should be transparent about how we’re spending public money.
The health and care system is complex. There are many different organisations involved. Being open about our ideas and proposals early on is a good way of getting the message out to these organisations, and starting a conversation with them.

One way in which we share our ideas openly is by blogging.

The document and drawing described above were drafted into an initial blog post by @evilstreak and @paul_furley, the drawing was redrawn by @demotive, and the resulting post is here – Booking is critical for transforming healthcare

My first User Research

Note – I originally posted this here on the NHS Choices blog back in February. It was written as three separate posts about User Research, from the point of view of someone who hasn’t been involved in this kind of thing before.

Over the last fortnight I’ve been observing User Research with my team, and it has been quite an eye-opener.

We’re at the point in our Discovery phase where we’ve made a bunch of assumptions about our users and their needs, and gathered information around these assumptions from various sources – on and off-site analytics, existing literature and research, social media, our service-desk tickets, and on-site surveys.

Now it’s time for us to talk directly to some USERS*

* Not all of the people we interview are necessarily users of the current NHS Choices service. Some of them might be potential users too.

User Research like this isn’t new to us. NHS Choices has had a dedicated Research team since 2007, but it’s in the last year or so that we’ve really started to more tightly integrate the work that the researchers do, into our delivery cycle. This is the first time we’ve involved the whole of the multi-disciplinary transformation team in observing and note-taking for the research sessions, doing the analysis and deciding on next steps within a couple of intensive research days.

Who do we interview?

For the two topics we’re focusing on right now – we’ve been talking to two distinct groups of people

Parents of children who’ve had Chickenpox in the last three months
People who sought a new Dentist in the last three months

We make sure we talk to a mixture of men and women from different socio-economic groups, of different ages, and with differing levels of internet skill.

We ask some quite detailed questions, so it’s good to get people who have had a relatively recent experience (hence the three month time-window) as the experiences they’re recalling will tend to be more accurate.

We use some dedicated participant recruitment agencies to source the specific people we want to interview. We supply a spec, like the parents described above, and they go and find a selection of those people. Obviously there’s a cost attached to this service, but the recruitment can be time-consuming, and it would be difficult to find a big enough cross-section of people ourselves. Outsourcing this to an agency frees up our researchers to focus on the actual research itself.

The setup

We do some interviews in the participants’ own homes – interviewing people in their own environment gives us a much better sense of how people look for information and where this fits into their lives. Also we get to meet participants who would not want to go to a viewing facility.

We also do interviews in a dedicated research facility – these are the ones that the rest of the team and I have been observing.

We’ve used a couple of facilities so far, one in London, and SimpleUsability in Leeds – just a five minute walk from our Bridgewater Place office.

Our interviews have been one hour long. The participants sit with a researcher – who conducts the interview – and a note-taker in the interview room. The note-taker might be another researcher or other member of the team – we’ve had UX Architects and service desk analysts taking notes in our sessions.

With the participants, the researcher and the note-taker in the interview room, the rest of the team are behind a one-way mirror with the sound piped in, observing the whole show.

And yes, with the one-way mirror, it fell to @seashaped and myself to make all the obligatory unfunny gags about being in a police interrogation scenario…

Interviewing

The interviews are based around a Topic Guide prepared beforehand by the researcher. This is based on input from our previous research, and includes specific subjects around which we want to learn more. The whole team feeds their ideas into the Topic Guide.

The interviews aren’t run strictly to the guide though – we’re talking to people about their lives, and the health of them and their families, so naturally the discussion can wander a little. But our researchers are great at steering the discussion such that we cover everything we need to in the interviews.

We decided not to put any prototypes in front of users in the first round of research. We’re trying to learn about users’ needs and their state of mind as they’re trying to fulfil those needs, so we didn’t want to bias them in any way by putting pre-formed ideas in front of them.

We did run a card-sorting exercise with users in the first round of Dental research – getting the participants to prioritise what would be most important to them when searching for a new dentist, by letting them sort cards.

We had a camera set up for the card-sorting exercise, so we could all see it clearly, without crowding around the mirror in the observation room.

As the interview takes place, the note-taker is busy capturing all of the insights and information that come up. As the participant talks, the note-taker captures each individual piece of information or insight on a separate post-it note. This results in a lot of post-its – typically we’ve been getting through a standard pack of post-its per interview.

GDS have written in more detail about some note taking good practices.

Sorting into themes

Once the interviews are over, we have to make sense of everything the users have told us. We have a whole load of insights – each one logged on an individual post-it note. We need to get from what the users have said, to some actionable themes, as quickly as possible, without producing heavyweight research reports. We use the affinity-sorting technique to help us do this.

This basically involves us sorting all of the post-its into themes. We’ve been having a stab at identifying themes first, and then sorting the post-its into those groups first. As the sort takes place we’ll typically find that a theme needs to be split into one or more themes, or sometimes that a couple of existing themes are actually the same thing.

This isn’t the job of just the researcher and note taker who conducted the interviews. The whole team that’s been carrying out and observing the User Research takes part in this process, shifting post-its around on the wall until we feel we have some sensible groupings that represent the main themes that have come out of the interviews.

Dental Affinity Sorting

Chickenpox Affinity Sorting

Although we’re not presenting our research findings as big research reports or presentations, we are logging every insight electronically. After the sort, every insight gets logged in a spreadsheet with a code to represent the participant, the date and the theme under which the insight was grouped. We’re reviewing our approach to this, but the idea is that over time this forms our evidence base, and is a useful resource for looking back over past research, to find new insights.

Hypotheses

Once we have our themes we have to prioritise them and decide what to do with them next. At this early stage this usually means doing some more learning around some of the important themes. We’ve been forming Hypotheses from our Themes – I think this helps to highlight the fact that we’re at a learning stage, and we don’t know too much for sure, just yet.

We’ve been playing around with the format of these Hypotheses. As an example, one of the strong themes from our first round of User Research on Chickenpox was around visual identification. We expressed this as follows –

We believe that providing an improved method of visual identification of Chickenpox

for parents

will achieve an easier way for parents to successfully validate that their child has Chickenpox.

When testing this by showing a variety of visual and textual methods of identification to parents of children who’ve had chickenpox

we learned …

So we will…

If you’re familiar with User Stories, you can see how this hypothesis would translate into that kind of format too. You could argue that all User Stories are Hypotheses really, until they’re built and tested in the wild.

Low-fidelity Prototypes

In order to test this hypothesis, we’re going to need some form of prototype to put in front of users. We’re working on a weekly cycle at present so we only have a few days before the next round of research. Speed of learning is more important at this stage, than how nice our prototypes look, so we’re just producing really low-fidelity prototypes and presenting them on paper.

For the visual identification hypothesis, here are some of the prototypes we’re presenting to users in our second round of research – see what I mean by low-fidelity, but this is just what we needed in order to explore the concepts a bit further, and learn a bit more.

We’ll base some of the questioning in our second round of research around these prototypes, and capture what we learn in our hypothesis template.

Based on this learning from the second round of research, we’ll either capture some user needs, write some new hypotheses to test, create some further prototypes to test, or maybe a mixture of all three.

Side effects

One interesting side-effect of our research sessions that we noticed was that some users were unaware of some aspects of our existing service, and as @kev_c_murray pointed out, some users left the sessions with an increased knowledge of what is available to them.

With comments like “Yeah I’m definitely going to go and look that up on your site now.” – we’re actually driving a little bit of behaviour change through the research itself. Okay, so if this was our behaviour-change strategy we’d have to do another 7 million days of research to reach the whole UK adult population, but every little helps, right…

What have we learned about how we do User Research?

The one week cycle of doing two full days of Research, then sorting and prototyping, is hard work. In fact it probably isn’t sustainable in the way we’re doing it right now, and we’ll need to adapt as we move into an Alpha phase.
Do a proper sound check at the start of the day – in both facilities we’ve used we’ve had to adjust the mic configuration during or after the first interview.
Research facilities do good lunches.
The observers should make their own notes around specific insights and themes, but don’t have everyone duplicating the notes that the note-taker makes – you’ll just end up with an unmanageable mountain of post-its.

More please

Lean UX Book We plan to do much more of this as we continue to transform the NHS Choices service. As we move into an Alpha phase, we’ll continue to test what we’re building with users on a regular basis – we’ll probably switch from a one-week cycle to testing every fortnight.

As someone from more of a Software Development background, I find it fascinating to be able to get even closer to users than I have before, and start to really understand the context and needs of those people who we’ll be building the service for.

If you’re interested in reading more around some of the ideas in this post, try Lean UX – it’s a quick read, and talks in more detail about integrating User Research into an agile delivery cycle.

Lego Flow Game

We run regular Delivery Methodology sessions for a mixture of Delivery Managers and other folk involved in running Delivery Teams. It’s the beginning of a Community of Practice around how we deliver.

One of the items that someone added to our list for discussion recently was about how we forecast effort, in order to predict delivery dates. Straight away I was thinking about how we shouldn’t necessarily be forecasting effort, as this doesn’t account for all of the time when things spend blocked, or just not being worked on.

Instead we should be trying to forecast the flow of work.

We’d been through a lot of this before, but we have bunch of new people in the teams now, and it seemed like a good idea for a refresher. My colleague Chris Cheadle had spotted the Lego Flow Game, and we were both keen to put our Lego advent calendars to good use, so we decided to run this as an introduction to the different ways in which work can be batched and managed, and the effect that might then have on how the work flows.

Lego Advent Calendar

The Lego Flow Game was created by Karl Scotland and Sallyann Freudenberg, and you can read all of the details of how to run it on Karl’s page. It makes sense to look at how the game works before reading about how we got on.

We ran the game as described here, but Chris adapted Karl’s slides very slightly to reflect the roles and stages involved in our delivery stream, and he tweaked the analyst role slightly so they were working from a prioritised ‘programme plan’.

Round 1 – Waterfall

Maybe we’re just really bad at building Lego, but we had to extend the time slightly to deliver anything at all in this first round! Extending the deadline, to meet a fixed scope, anyone?

The reason we only got two items into test and beyond was that the wrong kits were selected during the ‘Analysis’ phase for three items. The time we spent planning and analysing these items was essentially wasted effort, as we didn’t deliver them.

The pressure of dealing with a whole batch of work at that early stage took it’s toll. This is probably a fairly accurate reflection of trying to do a big up-front analysis under lots of pressure, and then paying the price later for not getting everything right.

It was also noticeable that because of the nature of the ‘waterfall rules’, people working on the later stages of delivery were sat idle for the majority of the round – what a waste!

Our Cumulative Flow Diagram (CFD) for the Waterfall Round looked like this –

You can see how we only delivered two items, and these weren’t delivered until 7:00 – no early feedback from the market in this round!

CFDs are a really useful tool for monitoring workflow and showing progress. I tend to use a full CFD to examine the flow of work through a team and for spotting bottlenecks, and a trimmed down CFD without the intermediate stages (essentially a burn-up chart) for demonstrating and forecasting progress with the team and stakeholders.

Round 2 – Time-boxed

We did three three-minute time-boxes during this round. Before we started the first time-box we estimated we’d complete three items. We only completed one – our estimation sucked!

In the second time-box we estimated we’d deliver two items and managed to deliver two, just!

Before the third time-box we discussed some improvements and estimated that we’d deliver three again. We delivered two items – almost three!

Team members were busier in this round, as items were passed through as they were ready to be worked on.

The CFD looks a bit funny as I think we still rejected items that were incorrectly analysed (although Karl’s rules say we could pass rejected work back for improvement)

The first items were delivered after 3:00 and you can the regular delivery intervals at 6:00 and 9:00, typical of a time-boxed approach.

Round 3 – Flow

During the flow round, people retained their specialisms, but each team member was very quick to help out at other stages, in order to keep the work flowing as quickly as possible.

Initially, those working in the earlier stages took a little getting used to the idea of not building up queues, but we soon got the hang of it.

The limiting of WIP to a single item in each stage forced us to swarm onto the tricky items. Everyone was busier – it ‘felt faster’.

We’ve had some success with this in our actual delivery teams – the idea of Developers helping out with testing, in order to keep queue sizes down – but I must admit it’s sometimes tricky to get an entire team into the mindset of working outside their specialisms, ‘for the good of the flow’.

Here’s the CFD –

The total items delivered was 7, which blows away the other rounds.

You can see we were delivering items into production as early as 2:00 into the round. So not only did we deliver more in total, but we got products to market much earlier. This is so useful in real life as we can be getting early feedback, which helps us to build even better products and services.

The fastest cycle time for an individual item was 2:00

A caveat

Delivering faster in the final round could be partly down to learning and practice – I know I was getting more familiar with building some of the Lego kits.

With this in mind, it would be interesting to run the session with a group who haven’t done it before, but doing the rounds in reverse order. Or maybe have multiple groups doing the rounds in different orders.

What else did we learn

* Limiting WIP really does work. The challenge is to take that into a real setting where specialists are delivering real products.

* I’ve used other kanban simulation tools like the coin-flip game and GetKanban. This Lego Flow Game seemed to have enough complexity to make it realistic, but kept it simple enough to be able to focus on what we’re learning from the exercise.

* Identifying Lego pieces inside plastic tubs is harder than you’d think.

Overall a neat and fun exercise, to get the whole team thinking about how work flows, and how their work fits into the bigger picture of delivering a product.