How to design a seven-day experiment that proves a GPT assistant reduces reopen rates without hurting csat

When I run pilots for new automation—especially AI that interacts directly with customers—I want one thing above all: clear, defensible evidence that the tech improves outcomes people care about. Recently I built a seven-day experiment to prove a GPT-powered assistant could reduce reopen rates...

How to design a seven-day experiment that proves a GPT assistant reduces reopen rates without hurting csat
Oct 15, 2026 • by Claire Moreau

Featured

Latest News from Customer Carenumber Co

practical framework for deciding when to automate vs when to route to a human in customer journeys

practical framework for deciding when to automate vs when to route to a human in customer journeys

When teams ask me whether they should automate a touchpoint or send a customer to a human, I always push back: the right choice isn’t binary. It’s a sequence of decisions guided by risk, value, frequency, and the customer’s context. Over the past decade I’ve seen the best outcomes come from a practical framework that combines data, customer empathy, and fast experiments. Below I share a...

Read more...

how to reorganize your team around customer outcomes instead of channels in four practical steps

how to reorganize your team around customer outcomes instead of channels in four practical steps

When I started helping support teams shift from channel-based organisation to outcome-focused squads, I expected resistance — but what surprised me most was how often teams hadn’t even agreed on the outcomes they were trying to deliver. Channels are easy to see: you can count phone lines, chat sessions, and inboxes. Outcomes are messier because they require judgement, measurement, and...

Read more...

a compact guide to choosing the right escalation workflow when you support regulated industries

a compact guide to choosing the right escalation workflow when you support regulated industries

Supporting regulated industries — healthcare, finance, telecoms, utilities, and similar — forces you to be precise about escalation workflows. I’ve built and reviewed workflows for teams operating under strict SLAs, audit trails, and privacy constraints, and I still lean on a handful of practical rules whenever I design or evaluate an escalation path. This compact guide walks through what...

Read more...

how to draft a cross-functional incident runbook that keeps customers informed and reduces escalations

how to draft a cross-functional incident runbook that keeps customers informed and reduces escalations

I’ve spent more than a decade helping support teams design processes that keep customers calm and teams focused during incidents. One of the most reliable levers I’ve found is a well-drafted cross-functional incident runbook: a living document that defines who does what, when, and how we talk to customers. Done right, it reduces escalations, shortens resolution times, and — importantly —...

Read more...

the minimal data collection plan that lets you demonstrate ROI from self-service articles

the minimal data collection plan that lets you demonstrate ROI from self-service articles

When I help teams prove the value of self-service content, the most common problem I see is an appetite for perfect measurement that never turns into action. Teams design complex tracking schemas, wait for months of noisy data, then decide measurement is "too hard" and revert to opinion-based decisions.I've learned to flip that script: start with a minimal data collection plan that answers the...

Read more...

how to design an onboarding support journey that increases trial-to-paid conversion by focusing on signals

how to design an onboarding support journey that increases trial-to-paid conversion by focusing on signals

When I work with product and support teams on onboarding, I stop thinking about "onboarding" as a single linear process and start thinking about a conversation made of signals. Every click, message, error, and delay is a signal that tells you whether a trial user is moving toward activation — or toward churn. Designing an onboarding support journey around these signals is the fastest,...

Read more...

how to build a one-week experiment to compare intercom flows vs current conversations

how to build a one-week experiment to compare intercom flows vs current conversations

I often get asked how to run fast, low-risk experiments that actually tell you whether a new automation or conversational flow is worth rolling out. Recently I needed to compare Intercom’s outbound flow/series approach with our existing live conversation model — and instead of a vague pilot, I designed a one-week experiment that gave clear, actionable results. Below I share the exact approach...

Read more...

what to measure to prove your chatbot is actually improving customer satisfaction

what to measure to prove your chatbot is actually improving customer satisfaction

I run a lot of experiments with chatbots and conversational automation, and one question keeps coming up: how do we prove the bot is actually improving customer satisfaction? It’s tempting to point to a single CSAT score and call it a day, but in practice you need a mix of quantitative and qualitative signals to build a convincing story. Below I share the set of metrics I always track, why each...

Read more...

the exact metrics to include in a leadership dashboard to get budget for a new support platform

the exact metrics to include in a leadership dashboard to get budget for a new support platform

When I’ve been in the room asking for budget for a new support platform, the difference between "we need better tooling" and "we deserve investment" always came down to one thing: the dashboard. Leaders don't buy feelings — they respond to clear, measurable impact. Here’s the exact set of metrics I put on a leadership dashboard to win budget conversations for a new support platform, how I...

Read more...

how to use conversational analytics to discover the three hidden reasons customers reopen tickets

how to use conversational analytics to discover the three hidden reasons customers reopen tickets

I used to treat reopened tickets as an annoying metric — a little blip on a dashboard that nudged managers to shrug and reassign. Over years of working with support teams, though, I learned that ticket reopenings are a goldmine. They reveal where conversations fail to resolve root causes, where automation promises but underdelivers, and where agents inadvertently create extra work....

Read more...