Go to app

AI completion score

Evaluates how effectively your AI assistant handles customer questions

Who can use this feature?

  • This feature is available to all users

What is AI completion score?

AI Completion Score is a metric that quickly evaluates how effectively your AI assistant handles customer inquiries.

It reflects your AI's problem-solving capability and helps you identify areas for improvement in your customer support automation.

This is calculated based on conversation data from the past 30 days, and the score is only displayed when there's conversation data available.

Understanding the visual indicators:

  • Less than 50%: Red progress bar with "Need optimize" badge

  • 50-80%: Orange progress bar with "Good" badge

  • Over 80%: Green progress bar with "Optimal" badge

How is AI completion score calculated?

The AI completion score uses the following formula:

AI Completion Score = (0.7 × RR + 0.3 × MRR) × 100%

Where:

RR (Resolution Rate) = (AI resolved conversations / Total AI conversations) × 100

  • AI resolved conversations: Number of conversations resolved by AI without human transfer

  • Total AI conversations: Total number of conversations where AI participated (including those later handed off to humans)

MRR (Merchant Review Rate) = (Unresolved questions reviewed / Total unresolved questions) × 100%

  • Unresolved questions reviewed: Number of unresolved questions you've addressed by adding a data source or marking as ignored

  • Total unresolved questions: Total number of questions AI couldn't answer, shown in the "Unresolved questions" section

Note: If you have zero unresolved questions, your MRR will automatically be 100%.

Why AI completion score?

The AI Completion Score provides valuable insights into your AI assistant's performance, helping you:

  • Identify gaps in your AI's knowledge base

  • Track improvements as you add more training data

  • Understand how well your AI is serving customers without human intervention

  • Prioritize which areas of your knowledge base need enhancement

How to improve your AI Completion Score

1. Increase resolution rate

🅰️ Manually review your Inbox and mark conversations as "Resolved" when AI successfully handled them

  • Go to Inbox

  • Go to the conversation and review

  • Click Resolve on the top right corner to resolve the conversation

🅱️ Enable Auto-resolve in Automation settings to automatically resolve conversations after a period of inactivity

  • Go to Settings

  • Go to Automation

  • Set up to resolve your conversation in Automatic resolution

2. Review unresolved questions

  1. Go to the Unresolved questions tab in the AI assistant section

  2. Review questions that AI couldn't answer effectively

  3. For each unresolved question:

    • Click Add answer to create a new data source entry

    • Or click Ignore for inappropriate or irrelevant questions

  4. Test your AI with the previously unresolved questions to confirm improvement

Learn more about Unresolved questions →

PreviousAI repliesNextWhat AI assistant can do

Last updated

Was this helpful?