Games and Learning

If there is one operation that understands the amazing possibilities of games to measurably teach students – and the limits on what is desired by schools right now – it is the nonprofit research and development studio GlassLab Games.

GlassLab Games released its educational version of Plants vs. Zombies in late June.

GlassLab has made a splash with a series of high-profile game launches – just last month there was their version of “Plants vs. Zombies,” a very slick game with NASA and the SimCity educational version that kicked off their public work. But more than those titles, the company has been focused on assessment, its literally in their name – Games Learning and Assessment Lab.

The games were built atop of a crazy-advanced assessment engine that allowed the game makers and teachers to access a veritable treasure-trove of data that could assess student behavior, gameplay and learning. It resulted in some of the most extensive research work into building assessments into games.

There was just one problem.

“What we realized was we were over-delivering on the assessment, plain and simple,” said Mat Frenz, the man behind partnerships and business strategy at GlassLab (as well as the group’s second employee).

Frenz added, “We spent thousands of hours and literally millions of dollars on these assessment models that we were building out and running efficacy studies against and all that… no matter how much we invested in it, we realized that the market wasn’t really asking for it.”

He later elaborated on how the assessment engine still was the medium-term plan for the organization, saying, “While we know that this assessment will be the foundation of incredibly powerful games in the near future, we realized that in order to take this work to scale, we needed to provide a broader set of entry points for teachers, and common data structures to streamline the integration process for developers.”

“So in addition to validating the assessment models and demonstrating learning impacts, we realized we also needed to invest in alternative approaches to capturing and surfacing assessment data to teachers.”

It says something about the current market for assessment that a powerful tool that could track every hover by a student and assess complex psychometrics of students was, perhaps, too powerful. But it also reflects an awareness of the reality facing those who want to unlock the power of games to teach.

And, in some interesting ways, it reflects the difficulty of bridging the game world and the classroom.

First, teachers don’t know what to do with the ocean of data that they could now access. And secondly, to adhere to the assessment engine, a game developer would have to abide by a fairly rigid game design and as Frenz is quick to point out, “One thing I know about game developers, what’s implausible is trying to export a game design philosophy to somebody who knows what they are doing.”

The Shift from Assessment to Data

And so the GlassLab team decided to create what they call a more “product tier” and to focus on data.

As Frenz explained, “Data was in service of assessment. We always had an assessment strategy. We didn’t have a data strategy.”

So what does the shift mean? The way Frenz explained it would be to think about Google and analytics.

“Google could have told you a design strategy for converting users,” he said, but they didn’t. They focused on a data strategy that correlated to that design strategy and then offered Google Analytics as a way to monitor the effectiveness of the site as measured by the data Google saw as important.

The result?

The sites adhered to many of the tactics Google recommended by allowing the developers to evolve their sites naturally and in a more scalable way.

“We need to do the same with learning game data,” he said.

GlassLab wants their new portal GlassLab Games to work the same way and so when Frenz took to the road to talk to 150 learning game producers his pitch was not start over with each developer.

“We kind of backed up on those assessment requirements and basically said, ‘hey, we have these reports. You just need to feed data into those reports.’ But as far as doing the efficacy studies and all the things that GlassLab does in this very deep design experiences – I mean, some of the places we pursued already had a base game, a good game that was being used to a lot of fanfare. Why force them to change it to make it effective. It’s already an effective game,” Frenz said.

The result is a suite of nine games – another 10 will be added by the end of the year – that offer teachers a single log-on and robust reporting that aligns to standards.

Developers will share revenues generated by the portal and by focusing on data rather than formal assessment requirements, GlassLab said they will be able to quickly iterate on the teacher reports to respond to changing needs in the classroom.

The game firms will also get access to the reports on student performance within the game, a step Frenz said is critical, “so they can watch the learning data. Then they can begin to tweak their game to make the data better and better.”

Working on Discoverability

The idea is to make it easy to bring games into the system to help producers access the potentially more regular flow of money schools may offer developers that is harder to see out of the hurly burly of the App Store. For Frenz, he said it is the natural next step in what GlassLab has been doing.

Once the firm started developing games they found building the assessment component “was getting really expensive and it was not very efficient. We ended up dedicating about two-thirds of our resources to building the infrastructure and about one-third to building the game.”

The other thing that GlassLab realized was what other developers needed.

“We heard from dozens of producers who were telling us, ‘I’m surviving on grants and App Store sales. I want to grow,’” Frenz said, saying the only real avenue open to them at the time was contracting with an educational publisher to build their game.

GlassLab has approached helping designers in a variety of ways, including sharing a lot of their lessons learn through videos and blogs like this explainer on Ratio Rancher:

But GlassLab stills sees its work, developing a single teacher and producer data system, focusing on games to teach in the classroom and trying to work as a commercial studio not an educational publisher, remains a risky proposition.

“There’s a reason nobody has ever done this and been successful and we are still working on it,” Frenz said. “It’s very clear after doing this for four years and talking to hundreds of teachers, hundreds of administrators, it’s very clear there is a desire there. It’s not clear that there is a demand. Can we get it as a budget line in a school district? That’s what we need to figure out.”

But he insisted the focus on products that make it easier for teachers to access fairly basic reports and for producers to easily connect their games to the GlassLab structure is not a turning away from the lab’s commitment to assessment.

“When I think about technology and how it grows, I think about evolution. It’s really difficult to leap-frog those evolutions and I think that was what we were trying to do with our initial approach to assessment. We were trying go 7-8-10 years into the future, without building that year 1, 2, 3 evolution,” he said.

“The shift from assessment to data structures I don’t think is a pivot. I don’t think it’s a turning away from our initial goal. I think we are just getting smarter.”