PostHog · AI product · Mid · 45 to 60 minutes

Should PostHog train on customer product data?

PostHog is a developer focused product platform for analytics, experiments, feature flags, and related product tools. Answer this product management case and get it graded line by line against a weighted rubric.

AI strategy
Product judgment
Privacy thinking
Trade-off analysis
Written communication

The situation

PostHog has started shipping AI experiences such as an installation wizard, an assistant, and an MCP server. Leadership believes the bigger opportunity is not just answering questions, but helping products act more proactively with less manual setup. You are the PM for this AI product area. The founders ask you for a recommendation on whether PostHog should invest in training models using customer product data for a narrow set of product tasks, or keep relying on general third party models and prompting. Why now: the current AI features are creating demand for better context and more tailored outputs, but using customer data for model training raises trust, privacy, and consent questions. Product data could be especially valuable for a specialized model, yet it can also feel sensitive, and a poor policy could hurt adoption or brand trust. The hardest issue is that model quality may depend on broad participation, while many customers say they prefer explicit consent for AI uses. You need to propose a direction that balances product ambition, customer control, and practical execution.

What to produce

Write a 1 to 2 page product recommendation memo.

Your answer must

Define one concrete first job for a custom model and explain why PostHog data could create an advantage for that job.
Recommend whether PostHog should pursue model training now, later, or not at all, and compare at least two realistic alternatives.
Propose a customer data participation policy, including the default, customer controls, and why that choice fits the product goal.
Describe a rollout plan that uses existing AI surfaces, such as the installation wizard, assistant, or MCP server, rather than a separate new product.
Specify success metrics and guardrails, including at least one product metric and one trust or consent related metric.

Constraints

Do not assume access to facts beyond the prompt and general knowledge of what PostHog is.
If you use any numbers, clearly label them as assumptions rather than facts.
Keep the recommendation within this product area decision, not a full company reorganization or legal rewrite.
Work within existing AI surfaces and product data, not advertising or unrelated data businesses.

Format

Use short sections with clear headings such as problem, options, recommendation, data policy, rollout, and metrics.

How PostHog actually played it

PostHog announces plans to train its own models on product data to power proactive, self-driving products, and explains why the data program is opt out rather than opt in. It places the idea in the context of its AI installation wizard, assistant and MCP server. A good case on the data and consent trade-offs of AI strategy.

Own data can justify training your own models for a narrow job
Be explicit about the opt out versus opt in choice and why
Tie model work to a clear product vision

Built from Training our own AI models by James Hawkins.