Context
Jad Naous, CEO of Grepr, comes from deep systems engineering and observability work. In this conversation with Martin Casado, he explains why AI has already changed software creation inside a 10-person team, while production operations remain much harder because real systems fail in messy, state-heavy ways. The discussion matters to PMs building developer tools, infrastructure products, or AI features because it draws a clear line between domains where AI is already compounding human output and domains where AI still needs strong signals, experiments, and human oversight.
The Big Idea
AI can now compress software creation dramatically, but operating live systems is a different problem class. The practical path is not to let a model guess from raw complexity, but to pair explicit signals with AI that can form hypotheses, run checks, and narrow causes like a strong engineer would.
That distinction explains both the near-term opportunity and the current limit. AI is already very effective when the problem is bounded and the success criteria are testable. It is far less mature when the system is open-ended, production-facing, and full of hidden state.
Key Insights
AI amplifies strong engineers
Jad's view is not that AI turns weak engineers into strong ones. It gives the most leverage to engineers who already handle complexity well.
- Strong engineers are better at breaking a large problem into smaller tasks, explaining the constraints, and spotting when the model is wrong.
- Weak or average engineers do not get the same multiplier because AI still depends on human judgment about architecture, trade-offs, and correctness.
- Jad saw the gap widen inside hiring and team work. The difference between an okay engineer and a very strong engineer grew by roughly 10x once coding agents entered the workflow.
Production code got faster
Grepr is only 10 people, but Jad described shipping production work at a pace that would have been hard to imagine a few years ago.
- One example was a data-lake auto-promotion feature that turned semi-structured log fields into queryable columns.
- Jad said that feature took about 10 days to ship himself.
- He estimates the same work would have taken about six months for one engineer in the past.
The PM lesson is that the cost of exploring technically ambitious ideas is falling fast. Roadmaps and staffing assumptions built for pre-agent development may now be too conservative.
Review shifts toward validation
Higher coding speed does not remove review. It changes what review should focus on.
- Jad described generating very large PR stacks with coding agents, then having humans review them.
- The review emphasis moved away from reading every generated line and toward checking the tests, golden expectations, design docs, algorithm choices, and how the pieces fit together.
- Jad also uses reviewer-style agents inside the workflow, but he still walks through the PRD, design doc, division of labor, and integration plan himself before letting implementation run.
For PMs, the implication is that AI-era quality systems should focus less on artifact polish and more on proof: benchmarks, tests, and clearly stated constraints.
Operations are not code
Martin kept pushing on a core distinction: code generation lives in a more bounded world than live operations.
- Writing code is closer to producing syntax from a training distribution that already contains enormous amounts of similar material.
- Running production systems means dealing with unpredictable workloads, failures, hidden configuration, human workarounds, and a huge number of possible runtime states.
- Jad agreed operations is a much harder problem, especially because incidents often involve not just software state but human context scattered across teams.
This matters for PMs evaluating AI product claims. AI for engineering is not one market. Coding assistance and autonomous operations have very different difficulty curves.
Start from change detection
Jad's strongest product point was that many AI SRE approaches start in the wrong place.
- A common workflow is: a metric spikes, an alert fires, then the system tries to backtrack through a complex environment to find root cause.
- Jad argues that this works poorly because root-cause search from a lagging symptom is too unconstrained.
- Grepr instead looks for what changed first. It scans very large log volumes, surfaces a new pattern or early warning message, and then lets AI investigate that narrower lead.
He described examples like a system message about connections running out, memory limits being exceeded, or buffers emptying. The message is not the answer. It is the earliest grounded clue.
Bounded ops will fall first
Jad drew a useful line between operational tasks that are close to solvable and those that are not.
- He thinks AI is very close to doing meaningful query optimization work because it can iterate against explicit workloads and performance benchmarks.
- He does not think AI can yet run a distributed cluster the way a great human operator can.
- The difference is whether the task can be grounded in measurable experiments and a constrained search space, versus an open-ended production environment with hidden state.
For PMs, this is a roadmap heuristic. Start with narrow operational jobs that have clear feedback loops before promising autonomous system management.
Mental Models & Frameworks
Code versus runtime behavior
Jad and Martin kept returning to a simple but important split.
- Code is the written instruction set.
- Runtime behavior is what the system actually does under real workloads, data shapes, and failure conditions.
- A system can have code that looks correct and still behave badly in production because latency, memory use, consistency, and failure handling only show up when the system runs.
Use this model whenever a team treats generated code as if it were equivalent to a working system. It is especially important for infrastructure, data, and reliability products.
Hypothesis-driven debugging loop
Jad's operating model for AI troubleshooting is not predict the whole system. It is investigate like an engineer.
- Step 1: detect a novel pattern or early warning in logs or telemetry.
- Step 2: inspect where that message came from in the code and whether it has appeared before.
- Step 3: generate several possible causal chains that could explain the signal.
- Step 4: run checks or experiments to see which causal chain matches the observed environment.
- Step 5: decide whether the issue is real, ignorable, or action-worthy.
This model is reusable for reliability and security. It reduces a massive state space by turning diagnosis into a sequence of narrower tests.
Extract signal before reasoning
Grepr's product architecture reflects a sequencing principle that many AI products miss.
- First separate signal from telemetry noise.
- Then feed the smaller, higher-value set of events into an AI system that can reason about them.
- Jad said Grepr started by reducing telemetry spend, sometimes by 90%, and then used the same strategic position in the data stream to power downstream troubleshooting.
Use this model when an AI feature is drowning in raw data. Better reasoning often starts with better filtration, not a bigger model.
Parallel agent orchestration
Jad's personal development workflow is also a reusable operating pattern.
- He uses Claude Code with several parallel sessions, usually three or four at a time.
- He has one main implementation thread, plus reviewer-style threads that act more like tech leads.
- He begins with a PRD and design doc, gets review on the plan, then asks the system to parallelize implementation where possible.
The takeaway is not the specific tool choice. It is that AI works better when the work has roles, checkpoints, and bounded context windows instead of one giant prompt.
Decision Principles
Principle: Match AI to bounds
When: deciding whether an AI feature should generate code, optimize a query, triage an alert, or run production systems autonomously.
Why: Jad's examples show AI is strongest when the task has explicit constraints, measurable outputs, and room to run experiments. The more open-ended the environment and the more hidden human context it contains, the less credible full automation becomes.
Principle: Validate behavior, not prose
When: reviewing AI-generated engineering work.
Why: Jad's team shifted review attention toward tests, benchmarks, expected outputs, and architectural fit. In complex systems, a plausible explanation or clean-looking code matters less than whether the running system behaves correctly.
Principle: Detect change before diagnosis
When: designing an AI reliability workflow or evaluating an AI observability vendor.
Why: Jad argued that chasing root cause only after a lagging alert fires is too unconstrained. A workflow that first surfaces a novel message, pattern, or environmental change gives the AI a far better starting point for diagnosis.
Trade-offs & Nuance
Speed raises review pressure
AI can collapse build time from months to days. That gain is real, but it shifts pressure onto review, testing, and integration.
- Bigger PR stacks arrive faster.
- Human reviewers become the bottleneck if quality systems do not evolve.
- Teams may need to review fewer implementation details manually and invest more in automated validation instead.
Detection beats blind reasoning
Jad is optimistic about AI operations, but not in a naive way.
- He is not proposing that a model read raw logs and simply know what is wrong.
- He is combining classical anomaly or novelty detection with an LLM that can reason after the signal appears.
- The trade-off is architectural complexity in exchange for a more grounded troubleshooting loop.
Hidden context still matters
Some production failures are visible in telemetry. Others depend on social or organizational state.
- Martin gave the example of a hidden deployment or some system fact that only one person mentioned informally.
- Jad agreed that incidents often involve knowledge distributed across people, not just machines.
- That means AI operations improves as more context becomes explicit in telemetry, runbooks, change history, and code, but it remains weaker when key facts live only in human memory.
Buyers prefer concrete savings
Grepr's go-to-market lesson is useful for PMs.
- Prospects were excited by the AI reliability story.
- But what got immediate pull was a simpler value proposition: cut Datadog or telemetry bills by 90%.
- Reliability gains were valuable, but they were harder for buyers to price quickly.
PM lesson: if a product creates both strategic and immediately quantifiable value, lead with the value that buyers can price fastest.
Common Mistakes
Mistake: Equating code with operation
A team can mistake successful code generation for successful system operation.
- Generated code may still fail under real workloads, data shapes, or resource limits.
- This mistake is especially dangerous in infrastructure products, where runtime behavior is the product.
- The better approach is to treat production behavior as a separate validation problem, with its own benchmarks and tests.
Mistake: Chasing lagging alerts
Starting incident analysis only after a metric spike or alert often produces weak AI results.
- The search space is already too large.
- The symptom may be several layers removed from the cause.
- Jad's alternative is to start from a concrete new pattern or early warning event, then reason forward.
Mistake: Assuming AI equalizes talent
Jad's experience cuts against the idea that AI makes engineering skill differences disappear.
- The best engineers still contribute the hardest parts: decomposition, constraint setting, trade-off judgment, and knowing when to distrust the model.
- If a team plans hiring or staffing as though everyone now performs the same, it may misread where leverage actually comes from.
Practical Application
Split build from run
For every AI initiative in developer tooling or infrastructure, separate three questions:
- Can the model create the artifact?
- Can the team validate the artifact?
- Can the system operate safely in production after the artifact ships?
This prevents a roadmap from treating AI wrote the code as equivalent to the feature works reliably.
Redesign reviews around tests
Before the next large AI-assisted engineering project, change the review checklist.
- Require a design doc before implementation starts.
- Require benchmark or load-test evidence for performance-sensitive work.
- Ask reviewers to focus on failure cases, golden outputs, and runtime regressions, not just code style.
This matches how Jad described reviewing large AI-generated PR stacks.
Triage novelty in telemetry
If your product includes observability or platform operations, add a lightweight workflow that flags genuinely new log patterns or early warning messages.
- Start with messages that indicate concrete change, such as new error classes, resource exhaustion signals, or never-seen-before patterns.
- Route those signals into an investigation flow before waiting for downstream incidents or KPI spikes.
- Measure whether the team catches issues earlier than the current alerting stack does.
Bound one ops problem
Pick a narrow operational task with clear feedback, such as query tuning, schema translation, or repetitive log triage.
- Give the AI real workloads or benchmark cases.
- State the performance target clearly.
- Let the model iterate against measured outcomes, not vague goals.
This follows Jad's argument that bounded operational tasks are the best near-term target.
Externalize hidden knowledge
Write down the system facts that senior operators currently carry in their heads.
- Recent changes and deployments.
- Known failure modes.
- Ownership and escalation paths.
- Environment-specific constraints.
AI and new team members both perform better when operational context is explicit instead of living in hallway conversations.
Questions to Consider
- Where is your team celebrating AI-generated output while still lacking evidence about runtime behavior such as latency, memory use, consistency, or recovery under failure?
- Which operational problems in your product are constrained enough for AI today, such as query optimization or log triage, and which still depend on hidden human context?
- If you reviewed the last three production incidents, what concrete environmental change, novel log pattern, or early warning signal appeared before the metric spike or customer escalation?
- What important facts about your production system still live in Slack messages, tribal memory, or side conversations instead of telemetry, runbooks, code comments, or change history?
Bottom Line
AI is already changing the economics of building software, especially for strong engineers working in bounded problem spaces. But running live systems is harder than writing code, so the near-term winners will be products and teams that combine explicit signals, disciplined validation, and AI-driven investigation instead of promising blind autonomy.
Resources Mentioned
| Resource | Type | Why it was mentioned |
|---|---|---|
| Follow Jad Naous on X: https://x.com/jadtnaous | Social profile | The feed description points listeners to Jad's writing and commentary. |
| Follow Martin Casado on X: https://x.com/martin_casado | Social profile | The feed description points listeners to Martin's posts and analysis. |
| Grepr: https://www.grepr.ai/ | Website | The company website for the product discussed throughout the episode. |
Case Studies Mentioned
Grepr's auto-promotion build
Jad described building a data-lake auto-promotion capability that turned semi-structured log fields into independent queryable columns. The work required handling metadata, commit behavior, and coordination across writers, which is exactly the kind of systems-heavy task that used to be slow. Jad said he shipped it in about 10 days, versus roughly six months for one engineer before coding agents.
Ignored warnings became incidents
Grepr has been running its troubleshooting approach internally and in some customer environments. Jad said there were cases where the system surfaced a signal, the team ignored it, and the issue later turned into a production incident. The lesson is not that AI solved operations end to end. The lesson is that early change detection may create a more useful starting point than conventional alert pipelines.
Observability cost hit limits
Jad said many engineering leaders were spending roughly 5 to 20% of infrastructure spend on observability. The deeper issue was not that they loved observability tools. They were really trying to buy reliability. As architectures became more distributed and telemetry volumes exploded, the marginal reliability gained per extra observability dollar flattened, which created room for Grepr's approach.
Tools & Products
| Tool / Product | What it does | Why it was mentioned |
|---|---|---|
| Claude Code | AI coding agent and development harness | Jad said he uses Claude Code as the center of his current software development workflow. |
| Windows Subsystem for Linux | Linux environment inside Windows | Jad uses it for development while working on Windows. |
| Datadog | Observability platform | Jad used Datadog bills as a concrete example of the telemetry cost buyers wanted to reduce. |
| Grepr | Telemetry reduction and AI troubleshooting product | Jad described Grepr as sitting in the data stream, reducing telemetry costs, and feeding higher-signal data into AI-driven investigation. |
People to Follow
Jad Naous
Jad Naous is the CEO of Grepr and a deeply technical systems engineer with experience across security, observability, product, and platform leadership. He is worth following if you care about the intersection of AI, distributed systems, reliability, and how engineering workflows are actually changing inside small technical teams.
Martin Casado
Martin Casado is a general partner at a16z and a longtime systems thinker with deep infrastructure and networking roots. He is especially useful to follow for first-principles views on AI, infrastructure markets, and the gap between software abstraction and operational reality.
Experiments to Try
Experiment: Query tuning copilot
- Hypothesis: an AI agent can improve query or translation performance when it is grounded in a real workload and benchmark harness.
- Change: give the AI a representative query set, the current execution plan or generated code, and explicit performance targets.
- Test: run the AI in a loop where it proposes optimizations, then re-run the benchmark suite after each meaningful change.
- Measure: latency, memory use, correctness regressions, and the number of human interventions needed before the result is shippable.
Experiment: Novel log shadow mode
- Hypothesis: surfacing never-seen-before production log patterns will detect incidents earlier than a workflow that waits for metric alerts alone.
- Change: create a shadow pipeline that flags novel error messages, resource exhaustion signals, or rare behavior patterns and sends them to an AI investigator.
- Test: do not auto-remediate at first. Compare the shadow pipeline's findings against incidents and alerts over several weeks.
- Measure: true positive rate, false positive rate, time-to-detection, and how often the novel pattern appeared before a customer-facing incident.
