Games and Learning

Over the past few months, we’ve shared some highlights of the literacy app analysis that we are conducting with New America to learn more about the apps that families and educators are using to help children learn skills such as alphabet recognition, phonics, vocabulary, and reading comprehension. We have been digging into the app store, looking at the lists of “top 50 educational” paid and free apps in the iTunes, Google Play, and Amazon app stores to examine the types of skills that these apps aim to develop. We know that browsing these lists is one of the primary ways that parents find educational apps for their children–but we know too that parents value reviews and recommendations from organizations.

Report “suggests there may be a disconnect between what parents buy for their children and what media products experts think are good for children.”

For this post we were interested in comparing the apps that appear in these top 50 lists to those awarded accolades by popular media rating groups, including Parents’ Choice Foundation, Common Sense Media, and Children’s Technology Review. To find these apps we collected lists of those that had won awards from each of these media rating organizations between 2012 and 2014, culminating in a list of 303 apps that had won at least one award from one of the organizations. Of this list, 66 apps (22%) targeted children between birth and age eight and had a focus on language and literacy development, and were thus included in our sample.

Finding 1: Very few of the popular paid and free apps won awards from top media ratings groups

Notably, 11 (or 17%) of the 66 award-winning apps with a focus on language/literacy were already in our sample as apps that were among the 50 most popular educational apps in the iTunes, Google Play, and/or Amazon app store during our sampling period.¹ These apps are listed in the table below.

We were perplexed to find so few of the award-winning apps among the Top 50 educational apps in the app stores during the two months of our sampling. This finding suggests there may be a disconnect between what parents buy for their children and what media products experts think are good for children. These differences may also reflect higher rates of marketing that producers of the most popular apps are doing to promote their products. Furthermore, it is not clear what particular algorithms are used by each app store to compile their Top 50 lists; this lack of transparency is also discussed in our first blog post.

There is an important caveat to this finding; namely, that these app samples were drawn from a particular point in time. We tracked the Top 50 educational apps across iTunes, Amazon, and Google Play stores over 8 weeks in February and March, 2014. The award lists represent apps that have won awards from 3 media rating organizations between 2012-2014. Thus, more of the popular apps may have won accolades from different organizations or at different times. Similarly, the other 55 award-winning apps in our sample may have ranked among the Top 50 in app stores during different weeks. Still, it is notable that most of the apps that were popular at a given time did not win awards from top media ratings organizations within the same year or several years prior.

Our additional findings, reported below, include analyses of the developers’ app descriptions we coded from app stores, and were conducted with our full sample of 181 apps.

Finding 2: Award-winning apps tend to cost a little bit more than popular paid apps

One striking trend we noticed early on was that the majority (87.9%) of award-winning apps were paid apps, rather than those freely available. This led us to wonder whether there were differences in average costs between the paid awarded apps and paid popular apps (i.e. “Top 50 Educational” in markets). We coded the cost of apps in categories from “less than $1.00” to “more than $10.00.” As shown in the figure below, on average, the awarded apps cost a bit more than the apps that were among the Top 50 educational paid apps in the app stores, although the difference was not dramatic ($1.00 – $2.00 for popular apps vs. $2.01 – $3.00 for award-winning apps, on average).

The reasons for this trend are not clear from our market scan, though we have a few hypotheses. It may be more resource-intensive to produce high quality apps, particularly to the extent that their creation involves the participation of education, child development, or literacy experts (as we describe in blog post 2). Thus, the higher cost to produce the highest quality apps, which in turn are more likely to win accolades from ratings groups, may be reflected in their relatively higher price. It may also be that parents are drawn to lower cost apps for their children’s use, and price is one factor driving the popularity of relatively cheaper apps in app markets. Price and popularity may also have a cyclical relationship. That is, if parents are largely drawn to free apps, then the high rates of downloads may boost those apps into the Top 50 lists in app stores, putting them before the eyes of parents who mainly search for the most popular apps.

In any event, these trends may have critical implications for which families download the highest quality educational apps. If the vast majority of award-winning apps are paid rather than free apps, and those paid apps are more expensive on average than other paid apps for children, it is likely that more children of parents with higher incomes will end up with the highest quality apps, compared to their peers. In fact, a similar trend—named the “app gap” for the gap in educational app ownership between higher- and lower-income U.S. families—has been borne out in 2013 research by Common Sense Media.

Finding 3: The most popular free apps are less likely to have the highest user ratings compared to popular paid or awarded apps.

Given that we used two separate sources to compile our sample of apps, we wondered further whether there might be differences in the user ratings of apps that were among the “Top 50” educational apps in the markets and those that were given awards by media-ratings groups. Some of the apps were available in multiple app stores, so to address this question we looked at each app’s mean rating across stores. With regards to the Top 50 paid and Top 50 free samples, we included only the ratings from the stores in which an app was among the top 50 educational apps (for example, if an app was among the top 50 free apps in iTunes and in Amazon then we averaged the user ratings from those two stores in the calculations for the figure below). As shown in the figure below, we found that Top 50 free apps were less likely than award-winning or Top 50 paid apps to have received the highest ratings from parents (4.5 or higher out of 5.0).

We plan to revisit this finding with additional data in order to shed more light on this trend. Anecdotally, our team has been finding that “free” apps often provide little content for free. Rather, they often consist of a “skeleton” or “snack pack” of the app with very limited content, and then promote the purchase of additional content within the app. It may be that this phenomenon factors into parents’ ratings of the apps, particularly if app descriptions are not clear about the extent of content provided in the free version of the app. That is, parents may download the app expecting a fully-outfitted app, and then are disappointed to find it contains very little actual content. As mentioned above, it is also possible that app price reflects quality to some extent and that parents tend to be less impressed by the quality of free apps.

Of note is the similarity between ratings of Top 50 paid and award-winning apps. It seems that, in general, parents are equally pleased with the “Top 50” paid apps as they are with the award-winning apps reviewed by media-ratings organizations once they download them.

Finding 4: The same language/literacy skills are common across popular paid, popular free, and award-winning apps

To follow up on analyses from blog post 2, we also looked for possible differences in the specific language/literacy skills that award-winning apps claim to teach, compared to the Top 50 paid and free apps. For this analysis we looked only at the most commonly mentioned skills, limiting our scope to those skills mentioned in the descriptions of at least 6% of apps. our analysis also indicates that the same few skills are among the most commonly targeted language/literacy skills across award-winning and top paid and free samples. In each of the three samples, teaching basic skills like alphabet/letter sound knowledge and vocabulary development were the most commonly encountered skills in app descriptions. Reiterating commentary from our earlier post, it may be the case that parents are seeking out apps that teach these things, or that apps just tend to teach these particular skills well (leading to their popularity with parents and with expert media raters). A related possibility, which should be explored with a larger future sample of language/literacy-focused apps, is that it is easier for developers to incorporate these relatively basic skills, or that perceived demand is driving their choices of what kinds of apps to design. For example, developers may perceive a greater market for apps for toddlers and preschool-age children compared to later elementary school.

However, the trends in the figure below suggest that award-winning apps are slightly less likely than popular paid or free apps to target seven of the eight most common skills. The exception was “learning to write or type letters,” which was relatively rare across all three of the app samples. These analyses examine the percentages of apps that mention each of the most common skills; as detailed below, we were also interested in how many different skills a given app claimed to target on average.

Finding 5: Award-winning apps tend to target only 1 or 2 skills simultaneously

Next, we computed a sum for each app of how many different skills were mentioned in its description. The findings from this analysis shed some light on Finding 3 above. As shown in the figure below, the majority of award-winning apps (61%) claimed to teach only one or two language/literacy skills based on their descriptions. This was the case for only 36% of top paid apps and 39% of top free apps, which were each more likely to target three or more different skills in a single app. What is more, the award-winning apps were most likely to mention at least one specific language/literacy skill in their descriptions; only 4.5% of award-winning apps did not mention language/literacy-learning at all or claimed to target only general language/literacy development with no mention of specific skills. Conversely, 9.1% of top paid and 15.3% of top free app descriptions did not mention specific language/literacy skills targeted by the apps.

It is not clear from these data why we are seeing these trends. It may be that educational apps that focus narrowly on just a few skills tend to be of higher quality than those that try to teach more skills simultaneously, or that reviewers at media-rating organizations feel that it is best to teach only a few targeted skills in a single app. It is also possible that parents tend to be drawn to apps that claim to teach a greater number of skills at once; and thus the top educational apps in app stores tend to have this characteristic, while the award-winning apps (most of which were not among the top educational apps in the stores) do not. This finding also highlights an important gap where more research is needed; to our knowledge, there is no available research indicating whether focusing narrowly on just one or two language/literacy skills in an app or aiming to teach multiple skills simultaneously is better for young children’s learning.

More in store for “What’s in Store”!

Taken together, these few early findings suggest that parents encounter different kinds of apps when they search for language- and literacy-focused educational apps for their children, depending on where they go to look for those apps. For example, parents who consult media-rating websites, like Common Sense Media, appear to encounter slightly more expensive apps than those who look to the “top” educational apps in iTunes, Google Play, or Amazon app stores. Those who are willing to pay for children’s educational apps, and browse the media-ratings sites or paid app markets, tend to find apps that have slightly higher user-ratings, compared to those who seek popular free language/literacy apps.

These particular codes represent only a subset of our market scan coding, and we look forward to releasing more insights in future posts. Currently, our team is engaged in our second round of coding of the same sample of 181 apps, which involves documenting aspects of the actual mechanics and content of the apps. Eventually we will be able to conduct cross-analyses with these two levels of coding, examining, for example, whether certain features are more common among award-winning or “Top 50” popular apps or vary by cost or user-rating. So please stay tuned for future posts and for the full report of our findings in the Seeding Reading book, to be published in 2015 with our partners at New America in Washington, D.C.

Editor’s Note: This post originally ran on the EdCentral and Joan Ganz Cooney Center sites. The Center is also responsible for overseeing gamesandlearning.org.

¹ Given that these apps were simultaneously top apps in the markets and highly rated by expert groups they are included in both samples in analyses comparing “Top 50” apps to “award-winning” apps.