Context

Aza Raskin invented the infinite scroll, co-founded the Center for Humane Technology, and runs the Earth Species Project (using AI to decode animal communication). His father, Jeff Raskin, started the Macintosh project at Apple and coined the "humane" in Center for Humane Technology. In this conversation with Rana el Kaliouby (host of Pioneers of AI, cross-posted into the Masters of Scale feed, recorded at the Masters of Scale Summit), Raskin makes a case every product builder should sit with: a designer's good intentions are close to irrelevant against the incentives a product enters, so the real work is anticipating what a technology will probably be used for, not just what it could ideally do. The first two-thirds is a clinic on design responsibility and incentives; the last third turns to AI decoding animal language, which is fascinating but less directly relevant to product work.

The Big Idea

The question to ask about a technology is not "is it good or bad?" but "are the incentives that govern how it gets deployed good or bad?" A product's intentions do not survive contact with the market; its incentives determine what it actually does at scale.

Raskin's own infinite scroll is the case study. He designed it to reduce friction for individual users, and it became a machine for extracting attention because that is what the incentives rewarded. Good intentions did not change the outcome.

Key Insights

New technology creates new responsibility

Raskin's central frame: every time you create a new technology, you uncover a new class of responsibility that did not exist before.

• His examples: we had no "right to be forgotten" until the internet could remember us forever, and no strong right to privacy until Kodak's mass-produced camera made it frictionless to capture anyone's image (which prompted the legal invention of privacy).
• Why it matters for PMs: a genuinely new capability in your product almost always opens a new category of possible harm that no existing norm or rule covers. Assume the powerful new thing you are shipping comes with a responsibility nobody has named yet, and go looking for it.

Confuse possible with probable at your peril

Technologists routinely reason about the best possible uses of what they build, while the market plays out the probable uses driven by incentives.

• The distinction: the possible is the ideal use case you imagine; the probable is what competitive dynamics and incentives will actually push the technology to do at scale.
• Infinite scroll: the possible was a smoother, more efficient interface. The probable, once an attention-capture incentive picked it up, was billions of people losing time (Raskin cites roughly half a million human lifetimes wasted scrolling each month).
• The takeaway: evaluate your product by the probable, not the possible. Ask what the incentives of the market it enters will force it to become, not what you hope it will be used for.

Incentives govern outcomes, not intentions

The reframe that ties the episode together: stop asking whether a technology is good or bad, and ask whether the incentives governing its deployment are.

• Why: individual good intent does not hold up under competitive pressure. Even a company that wants to do the right thing gets undercut if a competitor reaches the market faster by not doing it. Raskin's example: everyone can see that training an AI companion for engagement is more harmful than social media trained for engagement, yet the race pushes companies to do it anyway.
• For PMs: when you assess a feature or product, model the incentive structure it lives in, not just your team's intentions. The incentive, not the mission statement, predicts the behavior.

Consequences are unconsidered, not unintended

Raskin wants to retire the phrase "unintended consequences" and replace it with "unconsidered consequences." Yes, there are always hard-to-predict nth-order effects, but most harms were simply never examined. This shifts the responsibility: a consequence you could have foreseen with real effort is not an accident, it is something you chose not to look at. It makes anticipating harm a discipline you owe, not bad luck you suffer.

Designers exploit asymmetric knowledge

A quietly damning point about design ethics: designers hold asymmetric knowledge about how the human mind works, and that knowledge can protect users or exploit them.

• The mechanism: a "stopping cue" is the natural signal that tells you when to stop (an empty wine glass). Remove it (a glass that refills automatically) and people consume far more. Infinite scroll deliberately removed the stopping cue for content.
• For PMs: you often understand a user's vulnerabilities better than they do. That understanding creates an obligation. Using it to remove the user's natural stopping points is a choice to exploit, and designing those cues back in is a choice to protect.

The race for intimacy is coming

Raskin's framing of the next platform shift: social media was a race for attention; AI is a race for intimacy, to occupy the single most intimate slot in your life.

• The zero-sum trap: Reed Hastings joked that Netflix's competitor is sleep, because time is zero-sum. For AI companions, the competitor is human relationships, because any time you spend with a real friend is time you are not engaging with the product.
• Why it is more dangerous: an AI optimized for engagement can make you more dependent, more distrustful of other people, or more isolated, and it sits closer to you than a feed ever did. Sycophancy (the model buttering you up even when you say something harmful) is the visible early symptom of training for attention.

Clarity creates agency

Raskin's answer to "what can we actually do?" is that clarity creates agency: naming the probable direction clearly is what makes coordination possible, because people cannot coordinate against a threat they cannot see.

• The courage part: it is uncomfortable to be the person saying the train is heading the wrong way while everyone is enjoying the party, especially when you might be wrong. He cites Neil Postman's "clarity is courage."
• The hopeful part: big changes (women's suffrage, civil rights) felt impossible until they happened, then obvious. They came from many people taking many actions not visible to each other. For a PM, the actionable piece is that clearly articulating a risk, even when it is unpopular, is itself the lever that unlocks collective action.

Mental Models & Frameworks

Red team and yellow team

Raskin's concrete method for anticipating harm before shipping:

• Red team: figure out how bad actors could deliberately misuse your technology for harm. (Most teams know this one.)
• Yellow team: examine the harms that come not from malicious use but from bad and perverse incentives, including how the technology gets used outside the walls of your own company. (He credits Daniel Schmachtenberger for the term.)

The point of yellow teaming is that you cannot only ask "what can I do responsibly as one company," because the technology will be used far beyond your control. At minimum, name what those incentive-driven outcomes will be before you ship.

Ergonomics of mind and community

From his father Jeff Raskin: to build something humane, you have to deeply understand how people actually work, their "ergonomics." Get the body's ergonomics wrong and you build chairs that hurt people. Get the mind's ergonomics wrong (he calls it "cognetics") and you build systems that harm people psychologically. Get a community's ergonomics wrong and you break society. The rule that follows: the more powerful a technology becomes, the more harm it causes if you do not understand the human vulnerabilities it touches. Understanding where people are weak is what lets you protect rather than exploit them.

Reach up and out to fix the incentive

Raskin's model for escaping a race to the bottom, aimed at people with outsized influence. A single actor who does the right thing alone just gets out-competed. But a leader can "reach up and out": use their influence, resources, and relationships to help create rules that bind every competitor at once. If no platform could compete on engagement, the competition would continue on other dimensions, and the talent currently optimizing addiction would be freed for real progress. The competition still happens; it just stops undermining the whole.

Decision Principles

Principle: Judge by incentives, not intentions

• When: deciding whether to build or ship a feature that could be misused or could harm users at scale.
• Why: your good intentions will not survive competitive dynamics, and neither will a competitor's restraint. Model the incentive structure the product enters and ask what it will be pushed to become. If the incentives point toward harm, "we mean well" is not a safeguard.

Principle: Preserve the user's stopping cues

• When: designing engagement, feeds, notifications, or any consumption loop.
• Why: you likely understand the user's psychological vulnerabilities better than they do. Removing natural stopping points (as infinite scroll did) exploits that asymmetry. Deliberately designing in moments where a user can pause and choose respects it.

Common Mistakes

Mistake: Building for the ideal use case

The recurring failure Raskin describes is designing for the best-case scenario while ignoring what incentives will actually do with your work. He built infinite scroll imagining a cleaner interface, blind to how an attention economy would weaponize it. The better approach is to explicitly separate the possible (your hopeful use case) from the probable (what the market's incentives will force), and design and decide against the probable.

Practical Application

Run a yellow team before launch

• Do: in addition to red-teaming for malicious misuse, run a yellow team that maps the harms from bad and perverse incentives, including uses outside your company's control and what happens when competitors copy the feature.
• Why it works: most damaging outcomes are unconsidered, not truly unpredictable. Naming them before launch is the cheapest point to change course, and even documenting them creates accountability.

Separate the possible from the probable

For any significant feature, write two lists: the possible (the ideal uses you are designing for) and the probable (what the incentives of your market and competitors will likely drive it toward). Make the ship decision against the probable list. This turns a vague "could this be misused?" into a concrete forecast you can act on.

Design deliberate stopping cues

Audit your product's consumption loops for the natural stopping points you may have removed (an end to the feed, a pause before autoplay, a visible sense of "you are done"). Decide, on purpose, where users should be able to stop and reflect, rather than defaulting to whatever maximizes time-on-app.

Name the risk out loud

When you can see a probable harm in your product's direction, articulate it clearly to your team and leadership even when it is unpopular, because clarity is what enables anyone to act on it. Being the person who names the uncomfortable trajectory is the first step that makes changing it possible.

Questions to Consider

• What new class of responsibility does the newest capability in our product create, and have we actually named the harm it makes newly possible?
• For our next feature, what is the difference between the possible (the ideal use we are designing for) and the probable (what the market's incentives will push it to become), and which one are we deciding against?
• Have we yellow-teamed this, looking at harms from bad incentives and from uses outside our own walls, not just red-teamed it for deliberate misuse?
• Where in our product have we removed the user's natural stopping cues to increase engagement, and is that a choice we would defend if it were visible to the user?
• Do the incentives governing how our product gets used point toward user benefit or user harm, regardless of how good our team's intentions are?

Bottom Line

A product's intentions do not survive contact with the market, so the real question for any builder is whether the incentives governing its deployment are good, not whether the technology is. The practical discipline is to design against the probable rather than the possible, to yellow-team for harms driven by incentives (not just malicious misuse), and to preserve the human stopping points that engagement-maximizing design tends to strip away.

Concepts to Explore

The Platonic Representation Hypothesis

In the Earth Species work, Raskin describes how AI represents any language as a "shape" (embeddings), where words with similar meaning sit near each other, and how the shapes for different human languages, and even images and DNA, appear to fit inside one universal structure. The Platonic Representation Hypothesis is the idea that models are converging on a single underlying representation of how nature actually is. It is worth exploring for how it reframes what large models are really learning, beyond any single modality.

Transfer learning across domains

A striking result from the animal-communication research: training a model first on human speech and music makes it better at tasks on animal communication ("positive domain transfer"). The general lesson for anyone building with AI is that learning in one domain can improve performance in a seemingly unrelated one, which changes how you think about what data is relevant to a problem.

Notable Quotes

"We didn't need the right to be forgotten until the internet could remember us forever." (Aza Raskin)

"The fundamental question we need to stop asking is, is AI good or bad? Instead, we have to say, are the incentives that govern how AI is deployed good or bad?" (Aza Raskin)