Work / Child safety online · 2013–2016
Proactive risk detection for child safety online
Built proactive risk detection for child safety online in 2013, before platforms had Trust & Safety teams. The work moved from reactive case review to behavioral risk signals surfaced ahead of an incident report.
Designed new intervention and escalation pathways from detection to law enforcement referral, then led the development team that carried grant-funded programs from concept through deployment.
Detection is where the rest of the work starts. Two disciplines came out of it: designing for misuse and understanding the person causing harm well enough to see their next move coming. Both have applied to every regulated product since.
Program deployed · intervention and referral pathways operating
What’s in it
- The problem, in 2013
- Why it happened at an NGO
- What was built
- What a signal was made of
- Intervention as well as referral
- Three groups, and no two of them meant the same thing by success
- What the law allowed, and what the data had to meet
- You cannot assume good intent, so you design for misuse
- What it taught about who causes harm
- The team, and the funding that carried it
- What has changed since