Overview
The Agent Performance page helps to evaluate your support team’s volume by replies and tagged reviews. It provides both metrics by each team member and allows estimating the effectiveness of the automation in processing the reviews. The key indicators to monitor are the:
- Number of Replies
- Reply Effect
- Average Response Time
- Average First Response Time
- CSAT
- Reply Rate
Available Agent Data
On the Agent Performance page, you can select to view the data by All Agents, as well as compare the data separately by each agent. We show statistics for:
- AppFollow Agents (Team Members and Automation). Team Members who reply from AppFollow interface. Automation shows the data by Auto-replies (including the automated concern report) and Auto-tags.
- Email Agents. We show the name and email address of users who reply via any Email or Helpdesk Integration connected to the AppFollow account. Note: We show only replies statistics for Email Agents.
- API. Data for replies sent via the AppFollow API (methods #5. Review Reply, #6. Review Tags Update).
- Zendesk Agents. By default, we show the Zendesk App as a single user. You can set up an additional integration to see Zendesk Agents; this allows you to see the name and email addresses of individual agents who reply via the Zendesk App Integration. To see the data by replies and tags, it's necessary to add the Zendesk API key to AppFollow: Zendesk Agent Performance Integration
When you reply to a review from Zendesk, make sure to save the ticket assignee before sending a reply. This ensures that we will receive information about the assigned agent and show it in AppFollow.
Filters
Filters allow you to drill into the data you're interested in learning more about. Set up filters to:
1. Select agents. The list includes all agents who have ever responded to reviews.
2. Select an app. Here, you can select an app from the current workspace.
3. Select a time period for the analysis and the device country.*
On the Agent Performance page, the data is filtered by reply date (or tag date, report a concern date). It is different how data will be filtered on Reply to reviews and Reviews analysis pages.
4. Select additional filters:
- Workspaces;
- Review Language.
*The "Today" button is unavailable as we display data with a 1-day delay to avoid discrepancies and facilitate page performance. We display information for the day before yesterday if the current time you are looking at this page is earlier than 3 AM (GMT +3). If it is past 3 AM (GMT +3), you are able to see data from yesterday.
Summary
The first section provides the most valuable metrics to track when working with reviews in the app stores.
These metrics are:
- Replies submitted - the number of replies published in stores.
- Avg. reply time - the average reply time for the period.
- CSAT - the percentage of reviews with the positive reply effect (the reviews with the updated rating to more stars) from the total number of reviews with reply effect, except for the 5-star reviews.
- Reviews tagged - the number of tags applied.
- Reviews reported - the number of reviews reported as a concern.
CSAT
To calculate CSAT:
- We take the reviews with reply effect which rating has increased after the reply.
Note! Reply effect is calculated for reviews that were updated within 3 months after the reply. - We take the total number of reviews with reply effect.
- From this amount (total reviews with reply effect), we deduct the number of reviews with reply effect which have 5 stars before and after the reply.
- The formula is applied:
(1) Number of reviews with positive reply effect
_________________________________________________________________ X 100% = CSAT
(2) Number of reviews with reply effect - (3) 5-star reviews
Available metrics by each agent:
- Replies Submitted - the number of replies published in stores.
- Reply Share - the share of reviews with an agent reply out of the total number of reviews replied.
- Replies Per Day - the rate of reviews replied to the number of days. Shows how many reviews are replied to daily.
For the Replies Per Day, we only count days when there was at least one reply published.
- Average Reply Time - the average reply time for the period.
- Average First Reply Time - the average first reply time to a new or updated review.
- CSAT - the percentage of reviews with the positive reply effect (the reviews with the updated rating to more stars) from the total number of reviews with reply effect except for the 5-star reviews.
- Reviews Tagged - the number of tags applied.
- Tags Share - the share of reviews tagged by an agent out of the total number of tagged reviews.
- Reviews Reported - the number of reviews reported as a concern.
- Reply Effect - the update of the review rating as a result of an agent reply.
Reply Time
Reply Time shows how long it takes for a team to provide a first response (Average First Reply Time) and the Average Reply Time for the selected period.
- Average Reply Time is provided by default on the page.
- Average First Reply Time is the time between the first response to a new or updated review in the selected period.
Currently, the Reply Time shows the store reply time, which measures the entire duration from when a user submits a review to when the store publishes a reply. This includes the store's moderation time.
We are planning to update the reply calculation time in the future by separating the Store Reply Time from the Agent Reply Time. The Agent Reply Time will track how quickly an agent replies through AppFollow after the review is collected, focusing solely on your team's response speed and excluding store delays. To follow the progress of this update, you can visit the feature request page here.
Note: each column is a separate entity and does not include other columns (i.e. "< 1 day" does not include "< 1 hour" and "1 to 3 hours").
Share of replies to tagged reviews for the five most popular tags (e.g. only 34% of reviews tagged "Genie" have been replied to).
Reply Effect helps to estimate the impact of replies on the app review rating by an agent and the entire team.
Agent Reply Effect shows how many reviews were updated and how the review ratings changed as a result of replies by each agent. The comparison with the previous period provided.
All Agents Reply Effect shows the reply effect of all the replies (by team members and auto-replies) made during the period.
No Reply Effect the review rating changed and the number of reviews that weren't affected by replies.
Agent Workload
The graph shows how busy the team was during the selected period. Provides the data on Replies submitted:
Reviews Tagged on a daily or weekly basis:
And Auto-replies submitted that were moderated by a specific agent:
Report a concern
This section provides a Summary and Distribution of all reported concerns. Regardless of workspace settings, this page includes information for every report submitted, even those from different workspaces.
The Summary tab displays the number of concerns reported and the percentage of those deleted. It also breaks down the status of each report by category: Pending, Submitted, Error, Declined, Satisfied, and Deleted:
The Distribution tab allows you to filter and view reports by specific statuses or see all statuses at once, either on a daily or weekly basis: