Instacart's Blueberry AI Helps On-Call Engineers Tame Production Outages

When the Pager Goes Off, Every Minute Counts

If you've ever been the on-call engineer when the monitoring alerts start firing, you know the drill: your heart rate spikes, you open a dozen tabs, and you start digging through dashboards, logs, and deployment histories while the clock ticks. The worst part? The actual diagnosis often doesn't start until you've spent precious minutes just figuring out what's going on.

That's the problem Instacart set out to solve. The grocery delivery company just rolled out an AI assistant called Blueberry that lives right inside Slack, the same place their incident response already happens. The goal is to give on-call engineers a head start by gathering context and proposing root-cause hypotheses before they've even finished their first coffee.

Blueberry's Job: Handle the Boring Stuff

Blueberry isn't some flashy chatbot that writes poetry. It's built to do the grunt work that usually eats up an engineer's time during an incident. That includes figuring out which service owns the problem, reviewing recent deployments, scouring logs and metrics, searching internal documentation, and matching symptoms against historical incidents.

When an alert triggers, Blueberry spins up about ten sub-agents in parallel. They work through these tasks and then post a summary directly into the Slack thread where the engineer is already investigating. In about three minutes, the system can output an initial assessment—not a final answer, but a solid starting point.

Fourteen Years of Incident History as the Secret Sauce

Here's what makes Blueberry feel less like a generic AI toy and more like a tool that actually understands Instacart's infrastructure: it's plugged into the company's own incident history. Instacart has been tracking production incidents for over fourteen years, and that data is the backbone of Blueberry's reasoning.

That historical context appears to be paying off. In April, Blueberry handled roughly 25,000 diagnoses across more than 270 Slack channels. The company says its accuracy rate jumped from 60% to over 90% once the system was connected to that rich repository of past incidents. It's a good reminder that for AI in operations, domain-specific data often matters more than sheer model size.

Built to Fit the Workflow, Not Replace It

One of the smartest design decisions here is that Blueberry doesn't try to take over the incident response. It doesn't make changes to production, and it doesn't unilaterally decide what to do. Instead, it acts as a force multiplier for the humans on call.

Instacart's VP of Engineering, Siby Alappatt, put it this way: "Blueberry has proven to be a force multiplier, using AI to change on-call operations and help us quickly troubleshoot and mitigate complex issues in production." The system provides context and hypotheses, but the final call on diagnosis and remediation stays with the engineer.

How Blueberry Works Under the Hood

Under the hood, Blueberry is a set of AI agents connected to internal tools and data sources. It uses a persistent, tool-aware framework—think of it as a production path with a sidecar MCP (Model Context Protocol) catalog, plus persistent state and a review loop that helps it improve over time.

The key is that Blueberry can call tools to pull information from connected systems as it works. It keeps track of what it's already checked, so it doesn't repeat steps or lose its place. This kind of structured approach is crucial for production environments where you can't afford to have an AI hallucinate its way to a wrong answer.

Turning 25,000 Diagnoses into Lessons

Another standout feature is that Blueberry learns from every incident it handles. It retains knowledge from the troubleshooting process, merging past case studies with team-specific context. That means the next time a similar issue pops up, the team doesn't have to reinvent the wheel—they can build on what worked before.

The numbers are impressive: in a single month, Blueberry completed about 25,000 diagnoses, achieved a 99.9% workflow success rate, and made over 58,000 MCP tool calls. It's also adaptable—the system can be tailored to fit the working style of roughly 60 different teams across the company.

What This Means for SRE Teams Everywhere

Instacart's experience with Blueberry offers a few lessons for anyone thinking about bringing AI into their own operations. First, context is king. A generic LLM won't cut it; you need to connect it to your own incident history, service ownership data, and debugging signals.

Second, workflow integration matters. By embedding Blueberry into Slack, engineers can get help without jumping to a separate tool. That reduces friction and makes adoption much easier.

Finally, don't forget the human element. Blueberry is designed to augment engineers, not replace them. It gives them a head start and helps them work faster, but the responsibility for critical decisions still rests with people.

The Road Ahead

Instacart's CTO, Anirban Kundu, says Blueberry is part of a broader exploration into AI agent systems at the company. As these tools mature, we're likely to see more organizations follow suit, using AI not just for chat but for real, high-stakes operational tasks.

For now, Blueberry is a solid example of how to build an AI assistant that actually helps in the trenches. It's not about replacing the on-call engineer; it's about giving them a better starting point—and that can make all the difference when the pager goes off at 3 AM.