Catastrophic Imagination: How to Design When Everything is at Scale
Cheryl Platz showed up on a Saturday morning, and the first thing she did was make sure I knew she was a real human. I laughed because someone had sent me a video interview a few weeks earlier, and it turned out to be an AI agent. That person's AI agent spoke on their behalf. We're living in a time where we have to verify that someone is actually human.
That's the world Cheryl is thinking about.
She's a creative leader who's worked at Microsoft, Amazon, the Gates Foundation. She's taught at Carnegie Mellon. And she's been clear with herself about AI: she's mostly not using it, on purpose.
"I have stayed pretty hard away from generative AI for the most part, because of my concerns about environmental impact," she said. "I haven't found a tool yet that's like that I can feel comfortable about the provenance of."
What Cheryl brought to the conversation was a concept that stopped me cold. She called it catastrophic imagination. It's from her design philosophy, something she wrote about in her first book. And it's this idea that you have to imagine the worst thing that could happen if your product succeeds.
"You have to be able to both embrace like, okay, well, what? What if this thing is massively successful. What's the worst thing that could happen?" she said.
This isn't pessimism. This is responsibility at scale.
She talked about a video game that almost broke because the developers didn't imagine that tens of thousands of people would show up. The infrastructure couldn't handle it. Multiply that by a million when you're building AI systems. What's the worst thing that could happen if everyone uses this? What could break? Who could get hurt?
I kept thinking about something she mentioned almost in passing. The Twitter chatbot Tay. Microsoft released it, and it became so corrupted by bias that it was horrifying within hours. It removed humans from the loop, and humans were necessary.
"You shouldn't be removing humans from the equation. You can amplify humans. You can give them recommendations, but when you remove them, you get Tay," Cheryl said.
This is the core of it. AI can make recommendations. AI can flag patterns. But humans have to be the ones deciding, questioning, holding space for the unexpected.
What struck me hardest was when Cheryl talked about curiosity. She teaches at Carnegie Mellon, and the one norm she drills into her students is this: lead with curiosity.
"If you take a query with chat GPT, for example, right? Like chatgpt can give you an answer, but do you trust it? Where did the answers come from? Are you willing to chase the rabbit a little further down the rabbit hole?"
She's teaching students to not just accept what AI tells them, but to ask why. Where did that come from? Who benefits from me believing that? What else might be true?
This is the opposite of the frictionless interface. This is about making the process harder in exactly the right way, so we stay awake.
The thing that will haunt me from this conversation is when Cheryl mentioned the stories she's heard. "Some of the stories I've heard recently are bonkers," she said. "They sound like stuff you would have come up with about Skynet, but they're real."
She wouldn't share them on record. But she acknowledged it. Systems are already going off the rails in ways that sound like fiction.
And we're still pushing for speed. Speed over ethics. Speed over thinking. Speed over asking the people who might be harmed if we get this wrong.
When Cheryl talked about the decision-making in big companies, she pointed at something I've been sitting with. When people ask, "How do I make this faster with AI?" they're not solving the right problem.
"We're not necessarily solving the right problems. We're not asking the right questions necessarily. If you ask me, 'I need a tool to write my essay quicker,' you will get Chat GPT. If you ask the question, 'I need a tool that's going to help me be creative in a sustainable way that's not going to light the world on fire,' you might get a different answer."
This is about leadership. About being conscious enough to ask the real question, not just the easy one.
What I keep coming back to is Cheryl's commitment to thinking. She's willing to feel the tension of not using tools that everyone else is using, because she wants to stay awake. She's willing to teach in a way that demands rigor and curiosity from her students, even when they'd rather just get an answer from chat GPT.
This isn't anti-technology. This is pro-humanity. And it's the hardest work of all.
The future we're building will be a mirror of the people building it. Cheryl is trying to be the kind of person who builds with intention, with catastrophic imagination, with the willingness to slow down.
That's conscious UX. That's conscious leadership. That's the revolution we need, even if it's the slowest one.