Whenever I think of T-Mobile’s Quality Assurance process, I think of a cartoon I once saw on Kickassanime. It’s a picture of a Customer Service Representative and a Quality Assurance Representative with the caption: “The customer may have sounded delighted by your friendly demeanor and ability to fix the problem quickly, but I’m sure she was disappointed because you didn’t repeat her name 5 times.”
This cartoon illustrates so very clearly the problems with most QA systems. Anyone who works in a call center can’t help but shake their head and laugh in agreement. And if you work at T-Mobile, the comic strips on the website ring true in so many ways.
Throughout my time at T-Mobile, I’ve seen the QA system grow and change, some of it for the good of the reps and some of it for the bad, but constantly changing. It’s gone from having a team of specially trained QA reps scoring most of the calls, to coaches doing all the scoring. It’s gone from being 100-percent focused on T-Mobile policy and procedure, to be all about the customer’s experience, and then back to a supposed balance.
It’s gone from being one score per call, to being a four-part scoring system. The criteria for these scores went from following a zero-to-five ranking system to cutting out the numbers one and five, since it was difficult to determine the differences between a one and a two, and between a four and a five. Then, since it was still subjective, management added in expected behaviors, items that had to be done on every call in order to get at least a three, which was considered the business-standard. If you missed even one behavior, that automatically dropped you down to below business-standard. And if that wasn’t enough, they then cut out the business standard rating because, well, standard just isn’t good enough anymore.
Read More Tech Reads at Readesh
Take another step back. If it’s confusing for reps to understand and be able to adapt that quickly to a change, think about management scoring the calls! Since there is always going to be an aspect of subjectivity, T-Mobile tries to control this two ways: group calibration sessions and evaluating how accurate a coach’s score is to the standard. Sounds like a good plan to keep things on a level playing field so that reps don’t experience culture shock when going from coach to coach, right? Don’t let that fool you, it doesn’t help too much.
Calibrating calls is a wonderful idea, but there are a couple of problems. The calls are scored by a national team of QA personnel, various executives, and other leadership, but outside of listening to calls, these people have little to no interaction with customers apart from the most recent customer survey and very rarely have the in-depth knowledge of the day-to-day job the reps do. This can cause them to score a rep higher or lower than a coach might, simply because they don’t have that knowledge. This leads into the second issue: Coaches know that the calibration scores aren’t normally how they would score a call, so they score the calibration calls based on how the QA Calibration group scored similar calls in the past. Keep in mind that the goal is to be calibrated with National. I can’t tell you how many times when I’d sit down with my coach to score one of these calls, we’d look at each other and say something along the lines of, “Well, I think it’s X, but I know that QA will score it this way, so I’m going to put down Y instead.” We would often walk out of a session shaking our heads and wondering why in the world they would ever score a call that way.
The second safety measure is scoring the coaches on their scores. What happens here is National QA listens to 15 of the calls that the coach listened to and scored, then assigns their own score to the call. The coach’s Team Manager also follows the same process with two calls, and at that point, the variance is calculated to measure how far off from National the coach is. The problem here is that some coaches will score to National’s scoring style at the start of the month until they get all 15 reviews in, then will score how they think calls should be scored. And the rest? Well, some try to stay consistent with National and others score how they feel is appropriate.
All this doesn’t even begin to take into account the favoritism that runs rampant among the call centers. Reps follow certain coaches’ realignment after realignment simply because they know how that coach scores calls and therefore how to score high consistently. But that’s another story for another day!
