Most bug reports aren’t incidents. The problem is, engineers don’t know the severity until they’ve already spent time investigating.
When a bug report comes in, the first question engineers need to answer is: What’s the impact of this bug? Do I need to drop everything and look at it?
Ironically, in order to determine whether a bug needs immediate attention, they have to give it immediate attention. That pulls them away from whatever they were already working on. And even if the issue turns out to be minor, they’ve still paid the cost of switching context.
According to research summarized by UC Berkeley, recovery time after an interruption can range from 8 to 25 minutes depending on task complexity. So by the time an engineer determines that a bug report isn’t urgent, the interruption has already done its damage.
That was a constant struggle for engineers at Sibi until about a year ago, when we started using an AI-powered bot nicknamed Bug Ricardo to improve how we gather information and context.
Bug Ricardo acts as a first responder for bug reports: it gathers context, summarizes signals, and advises the team on what might be happening. But the important part is what it doesn’t do: it advises; it never acts. People still bring the judgment, empathy, and accountability that customer issues require.
If you’re interested in learning more about the technical implementation, you can read our previous blog post here.
Before Bug Ricardo
Imagine Chester, one of our engineers, is working on a bug that’s preventing some users from checking out when another bug report pops up: a user says they can’t sign in.
He’s worried there could be an issue preventing all users from logging in, which would be a “drop everything and fix” situation. So he stops what he’s doing and starts investigating. He discovers that only one user was impacted, and they were simply entering the wrong credentials.
By the time he gets back to his original issue, 20 minutes have passed. Time was lost debugging an issue with unknown severity.
The new way, with Bug Ricardo
Now, imagine the same login bug pops up, but this time Chester lets Bug Ricardo analyze the situation first.
Within a minute, Bug Ricardo reports that the auth logs show a mistyped password. Chester reviews the summary, decides the issue doesn’t need engineering intervention, and lets the reporter know they can follow up with the user directly. Then he stays focused on his original task.
Bug Ricardo didn’t solve the issue or make the decision for Chester. It surfaced the right context quickly enough for Chester to make the call without abandoning his work.
Why humans still make the call
AI is extremely efficient at analyzing large sets of data and providing insights quickly. But Bug Ricardo isn’t the sole decider — we use it to speed up our process, not replace human judgment.
People hold valuable domain knowledge. They have empathy for our customers, developed over time by understanding their needs and challenges. An LLM can present a path forward, but people bring curiosity, accountability, and the drive to get things right for the future.
That’s why Bug Ricardo only advises — it never acts. We leverage human judgment to make the final call, and using this tool has helped us get to that valuable point much faster.
Since we started automating portions of our bug report handling, we’ve decreased the time needed upfront to understand user issues. We’re able to respond to bug reports faster, without taking people out of the decisions that matter.
--
Say hi to Sarah 👋
Role: Software Engineer
Location: Denver, Colorado
Sarah Drew builds tools at Sibi that simplify the post-purchase experience. She's known for asking what a project is really solving before anyone writes code, and for carrying that answer all the way through to launch. When away from the keyboard, she loves to longboard dance and is currently learning to fish.
