Practice / Keeping users safe

What your users are up against

Your users need to be safe while they use your product. The people who know best what could harm them are the ones who have already been burned.

The time you got burned#

Most of us have been burned at least once, by a person, a business or a system we trusted. We still remember exactly how it happened.

The people using your product carry stories like that too, and so do the people your team will interview this month.

Safety is one of your users’ needs#

Building a product starts with the people it’s for: where they are, what they’re trying to do and what’s in their way.

Whatever the product does, those people also need to be safe and secure while they use it. So a team has to ask a second question: while somebody is using what we built, what could harm them, and who could cause it?

Harm grows out of circumstances#

A common first instinct is to look for a type of person to screen out. In the biggest study of workplace fraud, 87% of the people involved were first-timers and 88% had passed a background check.

In 84% of those cases, the person showed a warning sign while the fraud was going on. The most common were living beyond their means and money trouble.

So the threat to your users is mostly a set of circumstances: who can reach them through your product, under what pressure and with what access. The people who know those circumstances best are the ones who have lived through them.

People remember what burned them#

When somebody gets burned, physically, emotionally or in business, the feeling stays with them.

Memory research explains part of why the feeling stays. Emotionally charged moments are stored more strongly than ordinary ones, partly because of the stress hormones released at the time.

The feeling also comes back as a signal. The neuroscientist Antonio Damasio’s research describes the body marking an experience as good or bad. When a new situation looks similar, the same feeling returns, often before the person can say why.

So somebody who has been burned carries a detailed account of it: who was involved, what was going on and what they noticed too late. That account rarely ends up anywhere a product team will read it.

Ask about one specific time#

A question about one specific time brings out the circumstances, which are what a team can design for.

The psychologist John Flanagan published this method in 1954 as the critical incident technique, after using it to study pilots in World War II. Safety researchers still use it, and it works the same way in a product interview:

Can you tell me about a time you got burned, by a person, a business or a system you trusted?
What was going on in your life around then, and in theirs?
Looking back, were there signs before it happened?
Who else knew, or could have known?
Was there a product or system in the middle, and what did it let happen?
What would have made it harder to do, or stopped it sooner?

Listen for the story underneath#

The first telling is usually the event itself. Follow-up questions bring out what sits underneath: the circumstances on both sides, what had changed beforehand and what made the moment possible.

After a handful of these conversations, patterns start to show: the same pressures, the same kinds of access and early signs somebody set aside.

Put the team in the room#

Engineers, designers and leaders who hear a story firsthand reach the conclusion on their own, which makes it theirs to remember and act on.

Remembering what you worked out yourself has a name in memory research, the generation effect. In the 1978 experiments that named it, people remembered words they had worked out for themselves better than words they had simply read.

So one of the most useful things a leader can do is get people into those conversations, founders and executives included.

Ask with care#

Somebody telling you about a time they got burned has often carried it for years. They stay in charge of it: which story to tell, what to leave out and when to stop.

A person who senses judgment, of themselves or of somebody they know, leaves out the parts that matter most. So the questions stay on circumstances.

What it changes in the work#

The method fits inside research a team already runs.

Add one question about a time somebody got burned to your customer interviews
Bring an engineer and a decision maker to some of those conversations, so they hear the stories firsthand
Write down the circumstances from each story, and compare them across stories before you write any rule
For each new feature, ask who could reach your users through it, and what somebody under pressure could do with that reach
Read a background check for what it is, which is a look at somebody’s past

Back to the stories#

Those stories are already sitting with the people your team will talk to next. Asking for them is how a team learns what its users are up against.