Context
David Pawlan returns with Anish Acharya and Josh Elman after testing more than 147 AI assistants through Assistant Benchmark. The discussion is less about which demo looks coolest and more about what will actually make agents stick: trust boundaries, memory, judgment, shopping behavior, and new social patterns when agents interact with other agents. That matters to PMs because the hard part of agent products is shifting from raw capability to product design questions: what context to collect, what actions to allow, how much control to keep with the user, and what kind of value makes people come back.
The Big Idea
The next great consumer agent may win on trusted context plus judgment, not just automation.
The panel's view is that users will not hand over more of their lives just because an agent can click buttons. They will do it when the agent respects boundaries, handles ambiguous preferences well, and eventually feels like a personal edge, not just a task runner.
Key Insights
Trust blocks high-stakes adoption
- David said the biggest missing category is finance, even though money is an obvious agent use case.
- His explanation was trust. In group chats with AI power users, about 22% of more than 3,600 messages about pain points focused on trust as the main fear.
- The practical issue is not abstract safety. It is boundary failure. An agent that mixes personal and work contexts, or sends sensitive information into the wrong channel, can destroy trust immediately.
- The Shane Mack example made this concrete. A personal agent reportedly crossed into work and posted financial information into a company Slack. That is the kind of failure that makes users pull back even if the agent is otherwise useful.
Judgment matters more than autonomy
David's tests suggest the real product difference is not whether an agent can act, but whether it can balance messy human constraints.
- In a flower-ordering test, the prompt was simple but conflicting: deliver today and make it as cheap as possible.
- Instinct, Kicker, and Season were the strongest because they did not just find flowers. They handled both constraints at once.
- In a travel test, the prompt asked for a luxurious but cost-efficient weekend in Los Angeles. The agents all asked follow-up questions, but they interpreted the trade-off differently.
- Instinct balanced the two goals.
- Grokbot leaned more toward maximizing time.
- Dots leaned more toward minimizing cost.
- For PMs, this is a better evaluation lens than raw completion rate. Real user tasks often contain tension, not one clean objective.
Memory creates real switching costs
Long-term context is becoming a product moat, but not an unbreakable one.
- David said Dots was attractive partly because users felt it already had rich context from long-running chat history. In his survey of 354 AI power users, only 3% used it as their primary agent, but prior context was still the main reason people wanted to try it.
- That context advantage is not only about text. The panel discussed calls, screenshots, photos, purchase history, and ambient devices as future memory sources.
- At the same time, David tested whether users could leave. In his top 15 agent set, all but one product would export a memory file that could be imported elsewhere.
- The deeper lesson is that memory helps retention, but good portability can also become a feature. Users may trust an agent more if they know they are not trapped.
Social agents unlock new behavior
The most novel consumer behavior in the episode was not booking or shopping. It was agents creating new kinds of social interaction.
- Josh described playful, gossipy agents connected to friends. One agent told him a friend's power had been out all weekend, which led to a real human conversation that otherwise might not have happened.
- Those agents also compared different stories of how two friends first met, which created a fun shared moment rather than a productivity win.
- Wabi and Tomo point in a similar direction. Wabi organizes life into threads and group conversations. Tomo can place users with shared interests, such as LSAT prep, into agent-mediated group chats.
- David even ran an agent-only meetup in parallel with a human meetup. The results got strange after roughly 150 messages, but the experiment hinted at a real new surface: agents discovering links between people, goals, and opportunities in the background.
Paternalism breaks consumer agents
The panel strongly pushed back on the idea that agents should simply make users more rational.
- Josh's example was food. If an agent knows a user wants to lose weight, it might try to filter meal choices toward what the user should eat. That can feel helpful in theory and infuriating in practice.
- Anish made the broader point plainly: paternalism kills consumer products. Even a hint that the product is trying to improve the user against their will can create rejection.
- This matters because many agent visions assume optimization equals value. The panel's view was more nuanced. Users often want desire, aspiration, fun, or self-image protected, not corrected.
- That is why an agent that helps someone dress better or discover something cool may be more compelling than an agent that simply saves a few minutes.
Shopping may rewire distribution
Agent shopping could change which companies win, not just how consumers buy.
- In David's survey, 35% of respondents said they were already using assistants for Amazon plus retail shopping.
- The optimistic case is that agents search with infinite patience. They can find niche or direct-to-consumer products that a human would never discover through normal browsing.
- That could shift value away from aggregators and toward long-tail merchants or brands with strong products but weak marketing.
- The panel also suggested agents may develop their own proxies for quality, such as clean APIs or good documentation. That means agent-facing product quality could become a new form of shelf placement.
Reliability is still a moat
The episode pushed back on the claim that agent software is already commoditized.
- David said one of the biggest user frustrations is getting blocked or hitting outdated flows. If the agent fails on a task, users immediately wonder why they did not just use the browser themselves.
- He also sees sharp performance differences once products start scaling. Some agents look promising with small usage and then degrade under hundreds of thousands of users, with outages and weaker task completion.
- Tools like Hark and Kicker stood out partly because they invested in browser execution, direct integrations, and better visibility into what the agent was doing.
- For PMs, the lesson is simple: reliability, latency, and completion rate are still product features, not background engineering details.
Mental Models & Frameworks
Permission fences
The trust model discussed in the episode is not full freedom or full lockdown. It is scoped freedom.
- Give an agent clear boundaries around which accounts, apps, and channels it can use.
- Let the agent operate broadly inside that zone, but stop personal data from bleeding into work systems or public channels.
- This model is useful when a product needs enough autonomy to be helpful, but the cost of one embarrassing mistake is extremely high.
Multi-constraint agent evals
A useful way to evaluate an agent is to give it tasks with two or three legitimate objectives that pull in different directions.
- Example from the episode:
- Same-day flower delivery plus lowest price.
- Luxurious Los Angeles weekend plus cost efficiency.
- This kind of test reveals whether an agent can hold multiple goals, ask clarifying questions, and explain trade-offs.
- Use this model when simple pass-fail task completion hides the real product difference.
Memory portability onboarding
The discussion suggested a three-part memory model for agents:
- Acquire context: via chat history, calls, photos, screenshots, or purchase history.
- Make transfer easy: let users import and export memory so setup is fast and switching is possible.
- Avoid overfitting: distinguish durable preferences from short curiosity spikes, such as temporary interest in the World Cup or a child's favorite musician.
This model is useful when a product needs personalization without making users feel trapped or permanently mislabeled.
Secret superpower test
A strong consumer agent may need to clear a higher bar than time savings.
- Ask whether the product only removes chores, or whether it makes the user feel more capable, more connected, or more like the person they want to be.
- Josh's shirt example and the broader discussion about aspiration point to this test. Users may brag about an agent that upgrades their taste or helps them discover something special. They are less likely to brag about an agent that quietly saved five minutes on logistics.
- Use this lens when prioritizing consumer features. Delight and identity can be stronger retention drivers than efficiency alone.
Decision Principles
Principle: Scope access before action
- When: an agent touches sensitive domains such as money, messages, work tools, or personal photos.
- Why: the episode's trust examples show that one boundary violation can matter more than many successful tasks.
Principle: Benchmark conflicting preferences early
- When: an agent helps with shopping, travel, planning, or any task where users care about more than one outcome at once.
- Why: users rarely want pure optimization on one dimension. A benchmark that measures only task completion will miss the judgment failures users actually notice.
Principle: Optimize for real desires
- When: deciding whether an agent should guide users toward what is healthiest, cheapest, most efficient, or most rational.
- Why: the panel argued that consumer products lose people when they become paternalistic. An agent should meet the user's lived preferences before trying to improve them.
Principle: Use delight to build trust
- When: prioritizing early consumer use cases.
- Why: fun, social, or identity-enhancing moments can make users spend enough time with an agent to trust it later with higher-stakes tasks.
Trade-offs & Nuance
Control versus convenience
Users say they want autonomy, but the panel repeatedly came back to a softer version of agency.
- People want the agent to do the research, narrow options, and save time.
- Many still want final approval before a purchase, booking, or important message is sent.
- That means the best near-term product may not be fully autonomous. It may be a strong copilot with good checkpoints.
Privacy versus utility
More context usually means a better agent. It also raises the emotional cost of adoption.
- Photos, messages, shopping history, and ambient listening can all improve personalization.
- People may still hand over a surprising amount of data for small upside. The Hark free-pizza story showed users granting Google Workspace access for a modest reward.
- But when two products feel similar in usefulness, the more private one may win. That was the panel's core nuance.
Serendipity versus social risk
Agent-to-agent interaction could create entirely new social products, but the upside comes with awkward new failure modes.
- Social indirection can bring people closer. A playful agent can surface a fact or connection a person would never send directly.
- The same mechanism can also embarrass users, expose information too early, or violate norms in existing group chats.
- That is why the panel sounded more optimistic about new agent-native social spaces than about dropping agents straight into old iMessage-style threads.
Rich memory versus misreading
A richer memory system helps an agent know the user. It can also confidently learn the wrong thing.
- A burst of searches about the World Cup does not mean soccer is now a core identity.
- Researching a child's favorite artist does not mean the parent wants all future recommendations tuned that way.
- PMs need memory systems that can forget, decay, or ask whether a new signal is temporary or durable.
Common Mistakes
Mistake: Automating past user comfort
Teams may assume the best agent is the one that acts the most on its own. The panel's view was almost the opposite. Users often want autonomy only inside well-understood boundaries, with approval at the end. Shipping full autonomy before the trust model is clear can make the product feel reckless, not advanced.
Mistake: Treating curiosity as identity
Agents that store every search or chat as a stable preference will quickly feel uncanny. The episode gave simple examples: following the World Cup for a few weeks or exploring an artist because of a child. A good memory system needs ways to separate temporary exploration from enduring taste.
Mistake: Assuming infrastructure is solved
It is easy to believe better models erase execution differences. David's testing says otherwise. Browser control, integrations, latency, scaling behavior, and reliability still create visible product gaps. If users hit broken flows, they stop caring about the rest of the vision.
Practical Application
Map permission fences first
Before adding a new agent action, write down:
- Which data sources the agent can read.
- Which tools the agent can write to.
- Which actions require explicit approval.
- Which personal and work systems must never mix.
This exercise is often more valuable than debating one more feature. It surfaces the specific trust failures that would matter most.
Build ambiguity into evals
Create an internal agent benchmark with prompts that include competing goals.
- Ask for lowest cost plus fastest delivery.
- Ask for best quality plus strict budget.
- Ask for a message that is warm, brief, and professional at the same time.
Then score the outputs on trade-off handling, not just whether the task completed. That is closer to how users actually judge quality.
Let users import context
If your product depends on memory, reduce the cold start.
- Offer import paths from past chats, notes, screenshots, or purchase history.
- Let users review what was learned.
- Give users an easy export path too.
Easy portability can increase trust because users know they still control their data and preferences.
Add approval and rationale
For purchases, bookings, and outbound messages, show the user:
- What the agent chose.
- Why it chose it.
- What trade-offs it made.
- What will happen after approval.
This pattern keeps users in control while teaching them how the agent thinks, which is a trust builder on its own.
Test delight before autonomy
Prototype one feature that is useful mainly because it is enjoyable, social, or identity-enhancing.
- A group planning thread with friends.
- A style or shopping assistant that reflects saved tastes.
- A shared-interest matching feature around a real goal.
The episode suggests these wedges may build stronger daily habit than pure productivity tasks.
Questions to Consider
- Where in your product would a personal-to-work data leak cause the most damage, and what permission fence would prevent that exact failure?
- Which agent tasks in your roadmap involve real trade-offs, such as low price plus high quality or speed plus discretion, and do your evaluations measure those tensions directly?
- What memory sources would most improve your product, such as chat history, photos, screenshots, or purchase history, and which of those would your users actually trust you to hold?
- If your agent became better at making users rational, would your users feel helped or judged?
- What is one agent feature that would make a user feel they have a secret advantage, not just a faster workflow?
Bottom Line
The agent race is moving from capability to product judgment. The winners may be the products that combine strong boundaries, good memory, and human-like trade-off handling, then turn that foundation into delight, trust, and eventually a feeling of personal superpower.
Case Studies Mentioned
Hark's pizza permission trade
Hark ran a promotion offering free pizza in exchange for broad access to a user's Google Workspace. The panel used it as proof that many consumers will trade a surprising amount of data for even small utility, at least when the value is immediate and obvious. For PMs, the lesson is not that trust does not matter. It is that stated concerns about privacy often coexist with very permissive real behavior.
Agent-only meetup experiment
At David's meetup, humans talked in person while their agents joined a separate chat via QR code. The early prompts asked agents to introduce themselves, explain who their human was, and describe what help their human needed. After around 150 messages, some agents drifted into gibberish, but the experiment still surfaced a credible future pattern: agent networks that discover useful human connections in the background.
Tools & Products
| Tool / Product | What it does | Why it was mentioned |
|---|---|---|
| Assistant Benchmark | A testing effort across more than 140 AI assistants | David used it as the basis for comparing products on trust, judgment, memory, shopping, and task performance. |
| Instinct | A consumer AI assistant | It performed well in David's flower and travel tests because it balanced competing constraints instead of optimizing only one. |
| Kicker | An assistant with a Chrome extension | David highlighted it as strong on browser use and good at finding attractive flower options. |
| Dots | A personal AI assistant | It was discussed as appealing because users felt it had strong prior context from long-running chat behavior, even though only 3% of surveyed power users used it as a primary agent. |
| Wabi | A threaded, multiplayer assistant | Josh used it as an example of a different interface, one organized around life threads and group conversations rather than a single general chat. |
| Tomo | A social assistant with interest groups | It was mentioned for grouping users around shared goals, such as LSAT study, and potentially connecting them through their agents. |
| Scout | A shopping assistant | David called it interesting because it adds a swipe-style interface to learn shopping preferences in a more entertaining way. |
| Hark | An AI assistant with browser execution | It came up both for a strong browser visualization and for a pizza promotion that asked users for Google Workspace access. |
| Dazzle | A photo-based assistant from Marissa Mayer's new company | It was discussed as an example of using photo context while trying to forget photos that seem unnecessary or too sensitive. |
People to Follow
David Pawlan
Creator of Assistant Benchmark. In this episode he brings hands-on comparisons across 147 assistants, plus survey data and examples from active AI power users.
Josh Elman
a16z partner and longtime consumer product operator. Josh pushed the conversation beyond task automation toward social behavior, delight, and the idea that the best agents may feel like a private superpower.
Anish Acharya
a16z partner who framed many of the hardest consumer questions in the episode, especially around trust, social embarrassment, and why paternalistic products fail.
Notable Quotes
"What I've seen from 30 years in consumer product is paternalism will kill any product." (Anish Acharya)
"The number one fear that people talk about 22% of the messages are talking about trust as the main fear." (David Pawlan)
"When do these agents flip from getting something done to take it off our plate to giving us a secret superpower?" (Josh Elman)
"Can it help you become the human that you've always wanted to be?" (Anish Acharya)
