How to create Scorecards in Avoma

AI scorecards help you evaluate customer conversations based on specific rules and AI-generated scoring. This article is for admins or operators who want to standardize quality reviews without manual scoring.

Step-by-step

1. Open Scorecard settings

1. Go to Settings.
2. Select Scorecards.
3. On the Scorecard settings page, review the two sections:
   
   • AI scorecards
   • Manual scorecards
     
     

2. Start a new AI scorecard

1. Click Create new scorecard (top right).
2. Choose AI scorecards.
   

3. Add scorecard details

1. Enter a Scorecard name.
2. Add a short description explaining what the scorecard evaluates.

Keep descriptions brief and outcome-focused (for example, “Evaluates call quality and objection handling”).

4. Set auto-scoring rules

1. In Auto scoring rules, select:
   
   • A Condition
   • An Operator
   • A Value
     
2. Use the dropdowns to choose from the available options.

These rules determine when the AI scorecard is applied.

5. Add scorecard questions

You must add at least one question to create an AI scorecard.

1. Click Add question.
   
   
2. Choose one of the following:
   
   
   • Import from library
     
     
   • Add your question
     
     

Option A: Import questions from the library

1. Select questions from the available sections.

2. Click Import to add them to your scorecard.

Option B: Add your own question

Choose a question type:

• • Rating scale
  • Yes / No

• For Rating scale:

• • Click the empty question field and enter your question.

1. • Review the Examples of valid questions shown on the right for guidance.
   • Click Generate answers with AI to automatically create answers and scoring.
   • Review the generated answers and edit them as needed.

Note: You must add at least one rating option to a rating-scale question. You can add up to 5 options. Scores are normalized to 5 regardless of how many options you include.

• For Yes / No:

• • Enter your question.
  • Save to add it to the scorecard.

3. Add additional context

For both Rating scale and Yes / No questions, click Add additional context to give Avoma more information about what it should look for when evaluating the conversation.

Use this field to provide:

• Examples of phrases or behaviors that may satisfy the criteria.
• Different ways the expected behavior might appear in a conversation.
• Conditions or nuances Avoma should consider.
• Relevant customer, industry, or sales-process context.

Specific examples and context help Avoma account for different ways something may be expressed in a conversation and identify the evidence that supports the score.

4. Save the question when you're finished.

Repeat these steps to add multiple questions as needed.

6. Create the scorecard

1. Review your questions and scoring.
2. Click Create to save the AI scorecard.

7. Create a manual scorecard

1. Click Create new scorecard.
2. Select Manual scorecard from the dropdown menu.

8. Add manual scorecard details

1. Enter a Scorecard name.
2. Add a description.
3. Select the Conversation purpose from the dropdown.

9. Add questions to a manual scorecard

1. Type or paste your question into the question field.
2. Click Generate AI answers to generate suggested answers.

For Yes / No questions:

1. Enter your question.
2. Enter the answers.
3. Click Save to add the question to the scorecard.

Repeat to add additional questions as needed.

Troubleshooting and FAQs

 

What’s the difference between an AI scorecard and a manual scorecard?

An AI scorecard automatically scores the conversation based on the questions you configure and the auto-scoring rules you set up.

A manual scorecard requires a reviewer to score the conversation by answering the same set of questions manually after the meeting.

 

Why don’t I see AI-generated answers?

This is expected until AI generation is triggered.

• Confirm the question type is Rating scale.
• Click Generate answers with AI after entering the question.
• Check that the question is clearly written.

 

How can I improve the accuracy of my AI scorecard?

Give Avoma clear scoring criteria and enough context to understand what it should look for in the conversation.

• Define what qualifies for each rating or Yes/No response.
• Use Add additional context to include examples, variations, nuances, and conditions Avoma should consider.
• Include examples of how the expected behavior or topic might appear in a real conversation instead of relying on one exact phrase.
• Test the scorecard on a few representative meetings before activating it for your team.
• Review the reason and supporting conversation evidence behind each score. If a result does not match your expectations, refine the scoring criteria or additional context.

The more specific the criteria and context are, the better Avoma can identify relevant evidence in the conversation and apply your scoring rules consistently.

 

What’s next

• Learn how to apply AI scorecards to evaluations.
• Review best practices for writing effective AI scoring questions.

To drive adoption:

• Test the scorecard on a small set of interactions.
• Adjust questions and rules based on early results.