Games and Learning

New research indicates there’s little, if any, direct connection between rewards, such as achievements or badges, and what a player learns in an educational game.

New research indicates that in-game rewards don’t actually increase learning from a game.

Syracuse University’s School of Information Studies Associate Professor Jenny Stromer-Galley and a team of researchers developed two, original, Flash-based puzzlers to specifically measure whether in-game rewards helped players learn psychological concepts.

One game featured large number of rewards. The other did not.

She said that when they were tested, players from both games exhibited the same amount of learning.

Stromer-Galley said her team wanted to answer one key question: “Can educational games be more effective than other forms of teaching materials at teaching complex concepts like cognitive biases, and can it actually change behavior around decision making?”

The Syracuse professor and her colleagues corralled 365 undergraduate students from three different universities across the country to play the games.

“Although people in the high-reward game felt rewarded and they liked playing the game more than those who played a game with low rewards,” Stromer-Galley explained to gamesandlearning.org, ”the rewards had no effect on learning.”

The game, entitled “Cycles Carnivale,” strands players on an alien world. In order to find parts to fix their spaceship, they must earn currency by navigating a colorful carnival-like atmosphere, in either third or first-person perspectives, and playing traditional games like knocking over glass bottles with a ball.

Designers used this, and other similar carnival game scenarios, as a mechanic to judge a player’s decision to correctly determine what types of objects would be more effective to throw at the bottles.

They did this by using a combination of sound cues, collectibles and performances badges.

“Good games have good sound,” said Stromer-Galley, “and sound reinforces a lot of the contact.”

In the high-reward version of the game, a successful player would receive enthusiastic sound cues, and bronze, silver or gold coins based on their performance, similar to something like Angry Birds.

In the low-rewards version, player would receive only minimal sound cues, or informational text informing them they had chosen correctly, and no badges.

“Rewards is one game feature I think we are curious about, especially in the educational games world,” said Stromer-Galley, “because there is this hope that if you have the right sort, it might stimulate longer duration of game play, increase the amount of time people play with the game, and increase the likelihood of learning.”

But she said it’s important to point out the study did not look specifically at how rewards increase learning by incentivizing players to play longer. According to Stromer-Galley, the effectiveness of rewards, by themselves, had no direct impact on learning.

In other words, more rewards does not, by itself, lead to more learning.

“But what we do know, is that those who were in the high-rewards games did feel more rewarded,” said Stromer-Galley, “and said that they liked the game better.”

From a developers’ point-of-view, whether they’re trying to teach a student simple math, or complicated psychological perception biases, Stromer-Galley believes “you don’t need to go crazy with the rewards.”

She said balance is the key, and too many rewards may actually interfere with learning.

The Intelligence Advanced Research Projects Activity (IARPA), which is a division of the Office of the Director of National Intelligence, funded the research. The puzzle game isn’t available to the public because it was built purely for research, but Stromer-Galley said a commercially licensed version might hit the market in the future.