When people ask about our AI ticket triage implementation, they usually want to talk about the AI. The accuracy rates. The cost savings. The vendor selection. Those are interesting conversations.
But the more important conversation is about what we did first — before a single line of model configuration was written, before we signed a contract with a platform vendor, before anyone said the word “deployment.”
We rebuilt our entire training framework and restructured the support team into seven skills-based tiers. That work took longer than the AI implementation. It was less visible, harder to get executive attention for, and produced no dramatic launch moment. It was also the reason the AI worked when it went live.
This post is about that work. If you want the AI story first, it’s here: What I Learned Implementing AI Ticket Triage Before It Was Cool. But I’d suggest reading this one first — the sequence mattered, and understanding why will make the AI piece land differently.
Why Tiering Before AI
The premise of AI ticket triage is that a model can look at an incoming ticket and decide who should handle it. That sounds straightforward. In practice it requires the model to answer two questions simultaneously: what kind of ticket is this, and who is capable of handling it right now?
The second question is where most implementations run into trouble. If your support team is undifferentiated — if every agent is nominally capable of handling every ticket type — the AI has no useful signal to route against. It can classify the ticket correctly and still distribute it badly, because it doesn’t know that Agent A has deep expertise in billing issues, Agent B is still learning the product, and Agent C is currently at capacity.
Skills-based routing only works when skills are defined, documented, and tracked. That requires a tiering system. And a tiering system only works when your training framework is designed to move agents through it deliberately.
We had neither. So we built them.
The Seven-Tier Framework
The restructure defined seven distinct tiers, each with a specific skill profile, ticket type ownership, and training curriculum.
The tiers were not simply seniority levels. A senior agent who hadn’t developed expertise in a specific product area wasn’t automatically a Tier 5. The tiers reflected actual demonstrated capability against defined competencies — which meant we needed to define the competencies first.
For each tier we documented:
What ticket types the agent was authorized and equipped to handle independently. What ticket types required escalation to the next tier. What training had to be completed and assessed before a tier promotion was considered. What the expected resolution metrics looked like at that tier — not as a performance management tool, but as a signal of whether the agent was actually operating at the level the tier defined.
This specificity was uncomfortable for some people. It made capability visible in ways that vague seniority levels don’t. It also made development paths clear — agents knew exactly what they needed to learn to move up, rather than waiting for a manager to decide they were ready. The career pathing dimension of this connects directly to themes I write about on CXMaster’s Leadership & Career section — structured progression frameworks are one of the most underused tools in retention.
Rebuilding the Training Curriculum
The tiering framework was the structure. The training curriculum was the content. And the existing curriculum — a collection of product documentation, informal onboarding guides, and tribal knowledge passed between agents — wasn’t built to support either.
We rebuilt it from scratch around three principles.
First, every training module mapped to a specific tier competency. If a module didn’t directly develop a skill required for a tier progression, it didn’t exist in the curriculum. This sounds obvious. Most training libraries are full of content that exists because someone thought it was useful, not because it connects to a defined capability. The broader framework I use for training program design is covered in the Training & Enablement section on DataDrivenOps.
Second, the curriculum included assessment checkpoints, not just completion tracking. Finishing a module wasn’t the same as demonstrating the competency it covered. We introduced structured practical assessments — supervised ticket handling, scenario reviews, calibration sessions with team leads — that had to be passed before a tier promotion was approved.
Third, the curriculum was versioned and owned. Every module had an owner responsible for keeping it current. Product updates, workflow changes, and new ticket categories triggered a curriculum review, not a “we should update that eventually” conversation.
How This Enabled the AI
When we implemented AI ticket triage several months later, the tiering framework became the routing logic.
The model could classify an incoming ticket by type. The CRM integration provided availability data — who was logged in, what their current queue looked like, whether they were on a call. The tiering data told the model what each available agent was qualified to handle.
Put those three signals together and you have genuinely intelligent distribution: the right ticket to the right agent at the right time, without a human dispatcher reading queues and making routing decisions manually. The AI wasn’t just sorting tickets — it was allocating work across a structured team in a way that matched complexity to capability and respected capacity.
Without the tiers, the model would have had classification data and availability data. It would have known what the ticket was and who was free. It wouldn’t have known who was qualified — and in a support operation, those are not the same thing. This distinction is something the CXMaster piece on agentic AI covers well — the technology is only as intelligent as the operational structure underneath it.
The full metrics from the AI implementation — $200K saved, 89% accuracy, 54,821 correct predictions — are in the AI Ticket Triage case study. Those numbers were built on this foundation.
What Made It Hard
The resistance wasn’t to the concept. Most people understood why structured tiers made sense. The resistance was to the visibility it created.
A tiering system makes it immediately apparent when an agent’s actual capability doesn’t match the level they’d been informally treated as operating at. That’s useful information for a team lead. It’s uncomfortable information for the agent who’s been getting away with working above or below their actual level for a year.
Managing that transition required more care than the framework design did. We were deliberate about framing tier placement as a starting point rather than a judgment — everyone’s current tier reflected where they were now, and the curriculum gave them a specific path to move. For most agents, that framing worked. For some, it required more individual conversation.
The other hard part was curriculum ownership. Assigning owners to modules created accountability that hadn’t existed before. Some owners embraced it. Others needed regular prompting to keep content current. Building a lightweight review cadence into the calendar — quarterly for stable content, trigger-based for anything product-adjacent — helped, but it required ongoing attention.
The Sequence Matters
If I were doing this again, I’d make the same sequencing call: tiering and training first, AI second. Not because the AI couldn’t have been implemented without it, but because the AI would have been significantly less useful.
A classification model that routes tickets into a well-structured team compounds its value with every ticket. A classification model that routes tickets into an undifferentiated pool is a faster version of the same random distribution you had before.
The infrastructure you build before AI determines how much the AI can actually do. Most AI implementation plans skip straight to the model. The organisations that get the best results are the ones that did the unglamorous prerequisite work first.
For more on the AI implementation itself — the three-month training runway, the confidence threshold design, and the org change management — see What I Learned Implementing AI Ticket Triage Before It Was Cool.