AI & Technical question
A Fortune 500 customer asks for an audit trail for every AI-agent answer: which data sources were accessed, which permissions were applied, why the response was generated, and whether sensitive content was filtered or redacted. Design the admin and end-user experience, and the core APIs/data model needed to support this without overwhelming either audience.
- Glean
- AI & Technical
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests designing a dual audience (admin and end-user) experience plus the underlying API and data model for a detailed AI-agent audit trail, without overwhelming either audience.
How to approach it
- Define the data model first: for every answer, log data sources accessed, permission checks applied, redaction or filtering actions taken, and the reasoning path or retrieved context used.
- Design the admin experience as a searchable, filterable audit log, by user, by source, by time range, with drill-down into any single answer's full trail.
- Design the end-user experience as a lightweight, collapsed citation and source view by default, with an optional expand for the more detailed technical trail power users or auditors may want.
- Expose the underlying data via an API so security tools and SIEM systems can pull audit events programmatically, not just through the UI.
- Avoid overwhelming end users by keeping the default answer view unchanged, and only surfacing the full audit detail on demand.
- Pilot with one regulated customer's compliance team to validate the admin drill-down covers their actual audit requirements before general release.
What a strong answer includes
- Separates the data model (what's logged) from the two distinct UI needs (admin depth, end-user simplicity), rather than one UI serving both.
- Keeps the default end-user experience light, citations only, so audit depth doesn't clutter everyday use.
- Exposes an API for SIEM and compliance tooling, recognizing many admins want to pull this data into their own systems, not just view it in Glean.
- Validates the design directly with a compliance team from a regulated customer instead of assuming what fields they need.
Common mistakes
- Building one audit UI that tries to serve both casual end users and compliance auditors, satisfying neither well.
- Skipping an API and forcing all audit consumption through the UI, which blocks SIEM integration.
- Logging too little detail upfront and needing a rebuild once real compliance requirements surface.
Likely follow-up questions
- How would you handle audit data retention and deletion requirements?
- What's the minimum trail a smaller customer would still need, versus a regulated one?
More ai & technical questions
- How would you measure whether Glean's AI answers are accurate and well-cited?Glean · AI & Technical · Medium
- How would you design a governance model that behaves consistently across Glean's UI, public APIs, and heterogeneous enterprise connectors, for example, a shared policy layer for permissions, action scopes, audit logging, and redaction, even when source systems expose different authentication models and data granularity?Glean · AI & Technical · Hard
- Several enterprise customers report that the ServiceNow and GitHub connectors are ingesting data successfully, but search results and assistant answers still feel incomplete or low quality. How would you diagnose where the breakdown is across crawl freshness, permissions sync, metadata extraction, entity mapping, ranking, and UX, and how would you decide which fixes to ship first?Glean · AI & Technical · Hard
- Design the next version of Glean’s Indexing API and custom connector platform for large enterprises that need to bring proprietary data sources online quickly. What core capabilities would you prioritize, such as schema/modeling, auth and permissions, incremental sync, observability, and governance, and what principles would you use to keep the platform flexible without creating an inconsistent product experience?Glean · AI & Technical · Hard
- Glean Model Hub lets enterprise customers choose among LLMs for search, assistant, and agent workflows. How would you build a model-provider evaluation framework to decide which new providers to add, including the gating criteria, offline/online evals, and the tradeoffs you would make across answer quality, latency, security, cost, and enterprise-specific requirements?Glean · AI & Technical · Hard
- Suppose Glean wants to launch a new answer-generation experience in core search. How would you define the MVP and rollout plan given enterprise constraints around permissions, citations, latency, model choice, and admin controls? What would need to be true before you scaled it broadly?Glean · AI & Technical · Hard
More questions from Glean
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 1: Foundations: the model and the decisions it forces on you
- Chapter 8: Evals: define good and make the number defensible
- Chapter 6: Agents and agentic architecture