Blind updates

Most updates to mobile apps don’t make a noticeable difference.

The majority of updates to mobile apps don’t have a significant impact on user ratings, suggesting developers might release updates too frequently, according to a new study by UCL researchers.

They found free apps were less likely to make an impact when their developers release an update compared to paid apps, although among the impactful releases, the impact was more likely to be positive for free apps.

“There’s a culture among app developers of getting more releases out than your competitors, but our research suggests they should think more carefully before putting out a release, as it might bear very little benefit,” says Professor Mark Harman, Head of UCL Computer Science’s Software Systems Engineering Research Group.

The researchers analysed 26,339 app updates released on Google Play over a period of 12 months, from 14,592 apps which had all appeared in one of Google Play’s ‘Top 540’ lists at least once in the preceding year.

Using a novel causal impact analysis technique they developed, the researchers measured success by looking at user ratings and frequency of ratings.

They found that for paid apps, 40% of releases impacted subsequent user ratings, compared to only 31% of updates to free apps. Despite this, among the releases which made an impact, the free apps had more positive effects.

Among paid apps, the most successful releases came from the most expensive apps.

Their research also shed light on how releases can be designed to maximise impact. They found the most impactful releases offer new functionality and more descriptive release texts, but strategies should depend on the app’s popularity.

Among popular apps that aren’t quite in the top echelon, bug fixes resonate particularly well.

---