Practice / Designing for misuse

Close the pathway

Most misuse runs through steps a product already allows, taken from an ordinary account. Changing one of those steps can make the whole route stop paying.

An ordinary afternoon#

A manager thinks someone on their team is doing paid work on the side, and wants to find out without them knowing.

Every screen the manager will use to do it is one the product already gave them, and nothing about it will look odd to anyone.

Misuse usually starts from an ordinary account#

A lot of misuse comes from somebody already inside the product, on ordinary screens, allowed to see everything they’re looking at. Security work tends to picture somebody outside trying to get in.

Most features can be used for something they weren’t built for. A location share built to let a family know somebody got home can be used to follow a person who never agreed to it.

Eva PenzeyMoog, who wrote Design for Safety, designs with two people in mind: the person who might cause harm and the person it would land on. A team can keep both in the room the way it keeps a persona.

Write the sequence down#

The most useful single tool here is a list, in order, of every step somebody has to take. The criminologist Derek Cornish described this kind of list in 1994.

Once the steps are separate, each one is a place a team could have decided something else, so six steps give six places to act.

Written out honestly, the list usually has no hacking in it. Every step is something the product allows, so none of them would look odd to anybody reviewing it.

Security researchers now apply the same method to software. In a 2025 study of 19 online services, the method turned up routes that let one person keep access to someone else’s account. The owner had no way to take it back.

Which step to change#

Derek Cornish and Ronald Clarke grouped prevention techniques into five kinds: make it harder, make it riskier, make it pay less, wind people up less and take away the excuses. Product teams reach for the first three, because those look like security.

The fourth kind is where software does the most. Its list includes frustration, which the researchers define as having no options and no choices, and equipment that doesn’t work. So a product that frustrates the person using it is on a published list of things that can set harm off.

The fifth kind is about what a situation lets somebody tell themselves, and the screen is part of the situation. A button labeled adjust, where the honest word is overwrite, gives every person who clicks it a softer name for what they did.

Every fix has a cost#

The best fix takes the payoff out of one step and leaves the feature alone. The route stops being worth the trouble, and everything the feature was built for still works.

A second approver costs something every time, because somebody with a perfectly good reason now has to ask, and some of them will be annoyed. Naming that cost in the proposal is what lets the fix survive the first complaint.

Two common answers feel like progress and change very little: a notice, which hands the problem to the person already exposed, and a new setting, which that person has to find, change and explain if anybody asks.

Australia’s eSafety Commissioner puts the same idea in the first principle of Safety by Design: “the burden of safety should never fall solely upon the user.”

“They’ll just find another way”#

“People will just find another way” is the sentence that gets safety work cut. Researchers have measured it, so a team can answer it with a number.

In 2009, the criminologists Rob Guerette and Kate Bowers reviewed 102 studies that asked whether harm moves somewhere else when a situation changes. It moved in about a quarter of them. In slightly more than a quarter, places nobody had touched got better too, and across all 102 the totals came down.

So a fix buys a smaller total and more time before the next move. A proposal that promises to end the problem loses its budget the first time the problem comes back.

Plan for the next move#

The criminologist Paul Ekblom describes a back and forth that runs for years: a team prevents something, people adapt and the team prevents the adaptation. Car immobilizers led to bypass tools, and shop security gets worked around until the workaround is worked around too.

So every safety feature has a shelf life, and the useful habit is planning for the next move. Ekblom’s advice is to understand people well enough to see that move coming, which is the same job as understanding what a customer needs.

What it changes in the work#

Most of this fits into a design review a team already holds.

Write down, in order, every step somebody would take to misuse a feature, using only what the product allows
Change the one step where taking out the payoff costs ordinary users least
Name the cost of the fix in the proposal, including who will be annoyed by it
Read the screens for frustration, and for soft words that hide what an action really does
Put a date on each safety feature to look at it again

Back to the manager#

Once the step the manager was counting on changes, the route stops paying. Every screen still does its ordinary job for everyone else.