Games and Learning

Math Planet uses metrics and assessments to improve the games over time and inform future design work.

When discussion of assessment comes up around learning games it often focuses on gauging student understanding of a subject, but for Derek Lomas is it is a design concept and a business strategy.

Lomas is the PhD-holding brains behind PlayPower, a design firm that launched Math Planet, a suite of nine apps aligned to state standards, last month. 

 The team of designers has garnered support from the MacArthur Foundation and TechStars to help support its initial wave of games.

So, what make PlayPower’s approach different?

It may be the degree to which the entire firm is simultaneously developing games and conducting experiments that test design approaches and game mechanics.

Their suite of math apps aimed at teaching kids from grades 1-8 sits atop something called the Adaptive Machine Learning Technology.

Marrying Data and Design

The approach is a mix of human- and machine-run systems that are continuously evolving the games based on game play.

“We have an aspect of the adaptive nature of the product that is around individual performance – you know, it gets easier if you are struggling – but the other aspect adaptivity which is somewhat unique is how the product as a whole evolves based on how it is used,” Lomas said recently. “It is taking a whole kind of design space and testing out variations in it so whatever is working best in the game gets promoted.”

From the algorithmic side, the idea is as people play the game, the game itself adapts to use more effective aspects more often and to slowly weed out the “sub-optimal” designs, he said.

“The idea is we don’t know what is going to work best so we want to try out a bunch of different things,” Lomas explained. “Instead of building in a bunch of experiments and making changes we can build in the experiments and create the changes automatically.”

What surprised Lomas and other learning science researchers is what made it necessary to have a human components to their machine learning technology.

It turns out kids like to win.

“Kids really like things to be easy,” he said, adding this is unlike a lot of games that rely on hard-core gamers wanting to overcome a challenge.

Many casual games – whether for kids or not – rely on the gamer being able to relatively easily win or feel like they are winning.

“So sometimes this will result in game conditions that are much less effective for learning,” he said.

So, in addition to the automatic approach, PlayPower connects their designers to the data generated by the games’ assessments.

Lomas said this creates the opportunity to not only improve the current Math Planet games, but also helps them improve future projects.

“So we might see that there is a particular task that is taking people a long time to solve and they are dropping out there, well that seems like something we ought to deal with. And we probably don’t want to automate that, but it would benefit from a tight feedback loop in part because designers can learn why it didn’t work so well and then design better things in the future,” Lomas said.

This idea of connecting the games and the designers into the assessment of how the game works and how students perform is not just because the firm is filled with learning scientists. Lomas said it is all about the business side as well.

PlayPower hopes the data-driven approach will improve their chance of selling the nine apps to schools and attract future clients.

“The data-driven approach is all in the service of making a game that people are willing to pay for,” he said. “Everything we do now is sort of template so we can run these types of experiments and bring data back to the designers. It ends up being part of our infrastructure.”

But as much as the Math Planet games rely on tests to improve their designers’ work and the games themselves, the real test, Lomas admitted, will be in the market.

The market for math content is not overflowing with revenue opportunities.

According to PlayPower’s own analysis, the average amount of money spent on math content fro kids in K-12 is only $40 a year, and little of that is spent on digital tools.

“Given that digital content is way less than 1 percent of content spending there is still a lot of room to grow but in order to grow it really needs to prove that it is working,” he said, adding the data-driven nature of their project is part of the answer.

The firm hopes to see their apps find a market in schools across the country, having play tested in a variety of different schools, but whether the company can leverage this aggressive use of data to inform their design process into a workable business will have to be seen.