Test & optimize

Test and optimize how your AI assistant responds to customer questions

Who can use this feature?

• This feature is available for all users

Our AI assistant is still in beta version. If you have any concerns or feedback, please contact us to share and help us improve Chatty.

How to test AI assistant

AI test zone helps simulate customer interactions before going live, ensuring your AI responds appropriately to customer inquiries.

This helps you verify that the AI's knowledge and responses match your business needs.

Before AI assistant test

• Add your data (FAQs, product info, etc.)

• Add custom instruction for AI response (tone, voice, length of answer)

• Set up other settings (chatbot name, avatar, etc.)

Can I test AI assistant without enabling it?

You can still test AI assistant with data sources that you added without enabling AI assistant.

1

Go to AI assistant

2

Go to Test

3

In chatbox, ask and interact with AI assistant as a customer. You can use question examples to ask AI. (Click Regenerate questions for more examples)

Tips for better testing:

• Test one topic at a time

• Ask the same question in different ways

• Make sure responses aren't too long or too short

4

Check the responses and make improvements

What to check in AI response:

• Is the information correct?

• Is the tone friendly but professional?

• Are product details accurate?

• Are prices and availability correct?

5

Optimize based on test results

How to optimize AI responses

A. Unresolved questions

Unresolved questions are customer inquiries that your AI Assistant couldn't answer effectively.

When users click "Talk to a person" (reply button in AI settings), the question is listed as an unresolved question.

We only show unresolved questions in the past 30 days

How to optimize unresolved questions

1

Go to Unresolved questions

2

Check unresolved questions

What you should do with unresolved questions:

1. Look for patterns in unresolved questions to identify gaps in your training data

2. Group similar unresolved questions to create comprehensive answers that address various phrasings of the same inquiry

3. Use unresolved questions to identify new FAQs to add to your help center

3

Add answers to unresolved questions

Select a question from the unresolved list and click Add answer → Enter your answer.

Questions and answers will be added to data sources. (in Question list)

4

Test again

Go to Test zone and ask the question to ensure that AI assistant can handle that question now.

With duplicate or spam questions, you can click Ignore to delete the question from the list.

B. AI responses review

Each AI response now includes a "Review sources" button to see what information was used

This helps you understand why the AI responded the way it did.

Use this feature to:

• Add missing product information the AI should have included

• Correct inaccurate policy details

• Expand on partial answers that need more detail

• Add variations of common questions that weren't properly understood

What you should do

• Identify and correct inaccurate responses

  • When you notice incorrect or incomplete answers, click "Add relevant data"

  • You'll be redirected to the data sources page to add the appropriate information

• Watch for patterns in corrections

  • If you're frequently adding the same type of information, consider creating comprehensive entries about that topic

  • Group similar corrections into comprehensive answers to improve overall AI performance