Engineering · Beginner
Build an agent that triages your support queue
A contained, high-value first project: classify, route, and draft, without letting the agent send anything unsupervised.
Support triage is the best first agent project for most companies: the volume is high, the ground truth is clean, and the failure mode is contained because a human stays in the loop.
This playbook takes a team from nothing to a measurable time saving in roughly six weeks.
The playbook
01 Start with classification only
The first version does one thing: read an incoming ticket and assign a category and priority. It sends nothing and changes nothing else.
This is deliberately unambitious. It builds the data and the trust you need for everything after it.
02 Build the label set from real tickets
Take three hundred recent tickets and have your support lead label them. The categories that emerge will differ from the ones in your help desk configuration.
Disagreements between labellers are signal. They mark the categories that need splitting or merging.
03 Measure before you route
Run the classifier in shadow mode for two weeks: it predicts, humans decide, and you compare.
Only start routing automatically on the categories where shadow-mode accuracy clears your threshold. Route the rest to a human as before.
04 Add drafting, keep the human send
Once routing is trusted, have the agent draft a reply using your existing help articles as retrieval context.
The agent drafts, the human sends. This stays true for longer than most teams expect, and it is where most of the time saving already lives.
05 Close the loop
Log every edit an agent makes to a draft. Those edits are your highest-quality training and eval data.
Review weekly with the support team. They will spot degradation before any dashboard does.
Start before the gap gets expensive.
Tell us where the work is stuck. We will tell you, in plain language, what AI can and cannot fix, and what it would take to do it properly.