Chatty Help Center
Avada productsGo to app
  • Welcome
  • 🚀Getting started
    • Intro to Chatty
    • Quick start
  • 💬Live chat
    • Live chat
    • Your inbox
      • Conversations
      • Customers
      • Chat zone
      • Conversation details
    • Channels
      • Email
      • Facebook messenger/Instagram
    • Automation
    • Contacts
    • Team
      • Team member
    • Quick replies
  • 🤖AI
    • Overview
      • AI completion score
      • What AI assistant can do
      • How Chatty trains AI
    • Data sources
      • How to train AI
    • Settings
    • Test & optimize
    • Meet your new AI support
  • 🗨️Chatbox
    • Chatbox settings
    • Contact button
    • Deep links
  • 📱Mobile & web app
    • Mobile app
      • App tour guide
    • Web app
  • 📃Build FAQs
    • Add category
    • Add questions
    • FAQs page
    • FAQs block
    • FAQs analytics
  • ⚙️Others
    • General settings
    • Order tracking
    • Translation
    • Notifications
    • Analytics
    • Online hours
  • 🧩Integrations
    • Joy
    • Klaviyo
    • Air Reviews
  • 🔖PRODUCT ROADMAP
    • Product roadmap
    • Release notes
    • Pricing
  • ⁉️FAQs
    • Uninstall Chatty
  • 🔏Privacy policy
    • AI Compliance
    • Cookie policy
Powered by GitBook
On this page
  • What is AI completion score?
  • Why AI completion score?
  • How to improve your AI Completion Score

Was this helpful?

  1. AI
  2. Overview

AI completion score

Evaluates how effectively your AI assistant handles customer questions

PreviousOverviewNextWhat AI assistant can do

Last updated 24 days ago

Was this helpful?

Who can use this feature?

  • This feature is available to all users

What is AI completion score?

AI Completion Score is a metric that quickly evaluates how effectively your AI assistant handles customer inquiries.

It reflects your AI's problem-solving capability and helps you identify areas for improvement in your customer support automation.

This is calculated based on conversation data from the past 30 days, and the score is only displayed when there's conversation data available.

Understanding the visual indicators:

  • Less than 50%: Red progress bar with "Need optimize" badge

  • 50-80%: Orange progress bar with "Good" badge

  • Over 80%: Green progress bar with "Optimal" badge

How is AI completion score calculated?

The AI completion score uses the following formula:

AI Completion Score = (0.7 × RR + 0.3 × MRR) × 100%

Where:

RR (Resolution Rate) = (AI resolved conversations / Total AI conversations) × 100

  • AI resolved conversations: Number of conversations resolved by AI without human transfer

  • Total AI conversations: Total number of conversations where AI participated (including those later handed off to humans)

MRR (Merchant Review Rate) = (Unresolved questions reviewed / Total unresolved questions) × 100%

  • Unresolved questions reviewed: Number of unresolved questions you've addressed by adding a data source or marking as ignored

  • Total unresolved questions: Total number of questions AI couldn't answer, shown in the "Unresolved questions" section

Note: If you have zero unresolved questions, your MRR will automatically be 100%.

Why AI completion score?

The AI Completion Score provides valuable insights into your AI assistant's performance, helping you:

  • Identify gaps in your AI's knowledge base

  • Track improvements as you add more training data

  • Understand how well your AI is serving customers without human intervention

  • Prioritize which areas of your knowledge base need enhancement

How to improve your AI Completion Score

1. Increase resolution rate

  • Go to Inbox

  • Go to the conversation and review

  • Click Resolve on the top right corner to resolve the conversation

  • Go to Settings

  • Go to Automation

  • Set up to resolve your conversation in Automatic resolution

2. Review unresolved questions

  1. Go to the Unresolved questions tab in the AI assistant section

  2. Review questions that AI couldn't answer effectively

  3. For each unresolved question:

    • Click Add answer to create a new data source entry

    • Or click Ignore for inappropriate or irrelevant questions

  4. Test your AI with the previously unresolved questions to confirm improvement

Manually review your Inbox and mark conversations as "Resolved" when AI successfully handled them

Enable Auto-resolve in Automation settings to automatically resolve conversations after a period of inactivity

🤖
🅰️
🅱️
Learn more about Unresolved questions →