Reply Effect

Article author
Vera
  • Updated

Overview

Whether your team has only started replying to reviews or already has a well-established strategy of processing user feedback, it's important to see what effect this work has on your app's rating.

Users may update their review's rating for better or for worse after communicating with your team. However, app stores don't provide features for tracking these updates, so it's difficult to estimate your impact on the user's opinion.

AppFollow's Reply Effect algorithm helps you evaluate the real effect of your replies. It counts only those review updates that happened as a result of an agent's reply. The system takes into account:

  • The chain of agent replies and review updates
  • The period of time between the agent's reply and the user's review update

The Reply Effect is only available for reviews that are updated (rating and/or text) within 3 months of the reply.

In this article, you'll learn about the Reply Effect algorithm, how it's calculated, and how you can leverage this data across the AppFollow platform.


Reply Effect 

The Reply Effect is a vital KPI to create and adjust your reply-to-reviews strategy. The Reply Effect algorithm measures the effect the developer's reply had on the rating of the app.

The algorithm is based on 2 major factors: whether the review was updated after the developer's reply and how much time passed between the reply and the update. Below, you'll find some examples of these factors in action.

Factor 1: The review update took place after an agent's reply.

Example of a review update WITHOUT the Reply Effect:

In this example, the app user did not interact with the app developer. The review was updated independently of any interactions.

Example of a review update WITH the Reply Effect:

In this example, the app user updated their review after an agent replied to the previous version of their review.


A user may update their review's rating or they may update the text of the review without changing the rating.
The review will be considered as a review with the Reply Effect even when only the text was updated. The Reply Effect applies only if the review was updated after the agent's reply.

Factor 2: The review update took place within a specific timeframe – no later than 3 months after the agent's reply.

To optimize your reply strategy, it is important for you to be sure that the review was updated because of an agent's reply and not because of other factors. The Reply Effect has a time threshold that defines the range when an update is considered to have been impacted by a developer's response.
For a review to be included in the Reply Effect calculation, the update must occur within 3 months of the agent's reply. If more than 3 months pass between the developer's reply and the update, such updates are not considered to have a Reply Effect. 

If you need to customize this timeframe, please reach out to our Support Team or directly to your account manager.

Below is an example of a review update WITHOUT the Reply Effect, where the user updated the review more than 3 months after the developer's reply:

This example shows a review update WITH the Reply Effect, since the user updated the review the next day after the developer responded:

If the previous review was deleted, the rating update will still be calculated correctly, since AppFollow stores the review history.




Differences between filters on the Reviews feed

  1. With reply + Updated
    • Reviews that have a published reply and have been updated at any point (not necessarily after the reply).
  2. Reply effect
    • Reviews that have been updated (rating and/or text) within 3 months after receiving a reply.
  3. No reply + Updated
    • Reviews that have never had a published reply but have been updated at any point.
  4. No reply effect
    • Reviews that have never had a published reply.
    • Reviews that once had a reply but were updated 3 or more months later.
    • Reviews with a published reply, where the review was not changed afterward.

Where Can I Track the Reply Effect?

Agent Performance

The Agent Performance page provides insights into how your team is performing. In this section, the Reply Effect is counted for all agent-user interactions during the specified timeframe. This allows you to treat each interaction as a separate conversation and understand how an agent is performing in the context of an entire conversation.

For example, if an agent responded to a review 5 times, and a user updated this review after every reply, this review will be counted as 5 separate interactions and we will calculate the Reply Effect for each of them. This shows the actual effort an agent put into the review.

Unlike the Reviews Analysis page, the Agent Performance section takes multiple chains of reviews and replies into account to calculate the Reply Effect. If you need to evaluate the Reply Effect based only on the final result of the conversation, you can use the Reviews Analysis feature.

The Replies chart has been replaced by the Agent Performance page by April 2024. This change centralizes data for accuracy and convenience.

The Summary at the top of the Agent Performance page shows how many replies were submitted, the average reply time, CSAT of clients once the reply was submitted, number of reviews tagged and also reported:

On the Agent Performance page, you can also track the Reply Effect based on athe average change on ratings of the updated reviews after the agents reply, the average time it took for agents to send said replies and also the share of replies posted to reviews based on their tags.

On this page, we calculate the dynamics of every single reply and take the reply history into consideration. We calculate the Reply Effect for every update of the review (there may be several updates per review). This is why Avg. Rating Change on Agent Performance page can differ from the data on the Reviews Analysis page. It specifies how efficient the efforts of your Team are as opposed to eventual app rating dynamics displayed on Reviews Analysis.

The Reply Effect will help you understand how the replies are affecting your app rating and get insights on the optimization of your team efforts and strategy. Learn more about the Agent Performance page by checking this article from our Help Center.


Difference between Reviews Analysis and Agent Performance Pages

Understanding the difference between the Reviews Analysis and Agent Performance pages can help you track and measure your team's performance more accurately:

  • Reviews Analysis: This page displays all replies to reviews, including those made directly from the app store console. It provides a holistic view of all interactions regardless of where the replies were made.

  • Agent Performance: In contrast, this page only counts replies made specifically through AppFollow. It focuses on your team's interactions within the platform, giving insights into how AppFollow-driven responses impact user feedback and ratings.


Reply to Reviews

Apply the "Update" and "Rating change" filters to analyze how exactly the review rating changed. Click on "Show history" to see the details on review changes over time.


Was this article helpful?

0 out of 0 found this helpful