Power-ups don’t pass levels

The frame#

Sometimes the level calls for the cape. Sometimes the fire flower. Sometimes the mini mushroom, because it’s the only thing that fits the pipe. The power-up matters. The player still has to finish the level.

This is a useful frame because it puts the person back at the center of a conversation that keeps drifting toward the tool. A team that has adopted an impressive tool and isn’t getting past anything hasn’t made progress. It has acquired a power-up.

It isn’t the player’s job to know which power-up they need#

Reading the level is the product’s job. Handing someone a wall of tools and calling it choice asks them to do the reading I should have done for them.

This is where the AI conversation meets the older one about who carries complexity. Nobody should have to become an expert in model behavior to get the benefit of it, any more than they should have to become a database administrator to see their own records.

Four outcomes worth checking for#

When building and shipping AI, when in doubt, progress in any of these is usually a solid sign of being on the right track.

• Agency — the ability to choose, shape, or own the result
• Clarity — the ability to understand what has happened and what matters next
• Capability — getting better at the work, as opposed to bypassed
• Belonging — systems that recognize both the context and the contribution of the individual

Automation without progress#

The failure this test catches is work that got faster while nobody got better at anything, understood anything more clearly, or gained any say in the outcome. It looks like a win on a dashboard and feels like nothing on the ground.

Capability is the one most often traded away. There’s a real difference between a tool that makes someone better at their work and one that routes around them, and the second is usually cheaper to build.

What stays human when the record is evidence#

In systems whose output may be read by a regulator years later, some things shouldn’t be automated regardless of accuracy: the signature, the approval, the attestation, the judgment about whether a thing is actually safe.

That boundary is easier to name early than to reintroduce after somebody has automated an attestation.

Sources

• Nielsen, The Power of Defaults
• Safety by Design — eSafety Commissioner