AT&T implemented new AI tools and reaped the rewards. The company utilizes AI to categorize 40 million customer service calls annually. Initially using ChatGPT, which proved to be costly, AT&T developed a more affordable and quicker open-source AI system. This article is part of a series on AI adoption titled “How AI Is Changing Everything” across various industries.
Receiving 40 million customer service calls annually, AT&T encounters a mix of inquiries ranging from adding phone lines to reporting issues. Extracting valuable information from these calls proves to be challenging. While manually listening to each call may provide insights into emerging issues and prevent potential problems, the sheer volume of daily calls makes this an impractical task.
Previously, AT&T had automated transcription services but relied on manual sorting to categorize calls into 80 different categories for analysis. The ultimate goal was to address any issues promptly and prevent customer churn. With the introduction of large language models, AI can now efficiently ingest call summaries and categorize them.
Initially, AT&T used ChatGPT for this task but found it to be expensive and time-consuming. Seeking a more cost-effective solution, AT&T’s senior data scientist, Hien Lam, collaborated with Ryan Chesler from the open-source AI platform H2O.ai to develop a more flexible system. By leveraging a combination of smaller, open-source AI models with specific capabilities, they achieved comparable results at a significantly lower cost while maintaining data privacy.
The team distilled the larger GPT-4 model into three smaller, open-source models, each handling different levels of call complexity. By utilizing models like Danube and Meta’s Llama 70B selectively based on call content, they were able to reduce costs by 65% compared to ChatGPT while maintaining high accuracy levels and increasing processing speed.
“In our previous system, it took around five hours to process a day’s summaries,” stated Lam. They are now aiming to further increase the speed. “Since it currently takes 4½ hours for a full day, our goal is to achieve real-time processing after ending the call with AT&T,” Lam explained. “This way, we can obtain the outputs instantly.” Check out the full article on Business Insider.
