Games and Learning

The A-GAMES project, which released a national survey of teachers’ use of games in the classroom last year is now out with a report that offers deeper insights into what tools help teachers incorporate games into the classroom

The report, published late last week, examined how a range of different features – from dashboards for teachers to lesson plan options – affected the usefulness of the games as a teaching tool.

“Our report is a call to the game development community about where they might focus next to make educational games even more useful,” said Barry Fishman, a professor in the U-M School of Information and the School of Education.

Fishman and his fellow researchers Michelle Riconscente of GlassLab Games and Jan Plass of New York University reported on in-depth interviews and observations of 30 teachers in the New York area and found some surprising results.

“The most surprising finding for me was that the most common mechanisms in games for reporting progress – things like points and stars – are not that useful for teachers,” Fishman said. “For many of the teachers, it was hard to tell from these progress markers what the students were learning. So a student has 100 points. Does that mean they are learning addition?”

Fishman studied mostly games that could be easily incorporated into the classroom setting – ones that were more like exercises versus the longer, more immersive games, but he said he found an increasing appetite among teachers for both types of experience.

“I perceive a general movement that combines games with data-driven tools for teaching, such as learning analytics, digital badges, and other approaches to personalized learning,” Fishman told gamesandlearning.org this week. “These are all buzzwords at the moment, but we are starting to see a convergence that brings them together into useful and productive learning environments.”

We asked him a series of questions about what type of features worked best from the teachers’ perspectives and what developers ought to take away from this study as well as the earlier national survey of teachers’ use of games in class.

gamesandlearning.org: You mention that many of the games have features that could be used to help teachers assess student performance – stars, scores, points – but that teachers often don’t know how to translate those features into assessments for class. Can you talk a little about why this is? Do teachers want an explicit scorecard that reports a student’s proficiency or is it more general then that?

Barry Fishman: We were a bit surprised by this finding ourselves, but upon reflection is started to make more sense to us.

The issue is that a “score” (or other indicator of progress) in a game means something within the game itself, but how that translates to “real world” progress in learning can be harder to discern. I think what teachers were looking for is some kind of a translational guide between the language of the game and what this might mean for learning.

I should note that many games do attempt to provide this kind of information, but for the teachers in our study it was not prominent enough. Our study is less about what game designers intended and more about how those designs were received or interpreted by teachers.

gamesandlearning.org: Can you describe how teachers in your case study think about feedback like dashboards? Do they want a simple level of achievement – like a grade on a test – or are they looking for something else?

Barry Fishman: The question of dashboards is similar to the one of interpreting points or scores. But in this case it is about learning both across the students in their class and across multiple game or learning experiences over time. For many games, there is no aggregated or class-wide reporting system, so this means teachers have to find ways to look over students’ shoulders or otherwise collect progress reports from students. (Screen capture tools turned out to be useful for this – in our study this functionality was provided by BrainPOP’s SnapThought tool.) But imagine a situation where every activity a teacher does has a separate grade book; that would be pretty inconvenient when it came to tracking progress across time and across different activities.

I think teachers would appreciate dashboard information that helps them measure students’ progress toward learning goals for the course, and not just report a single grade or score. Knowing how students vary in their understanding of different concepts would also have value. Though they were not part of our study, systems like Manga High and Khan Academy have particularly rich dashboard features that do just this. However, they also control the design and production of all the activities, enabling this kind of integration. A portal like BrainPOP is aggregating games made by many different developers, and there is (as of yet) no universal standard for reporting the kind of information the teachers in our study wanted to see.

gamesandlearning.org: You documented a series of different feedback mechanisms – from stars to dashboards to screen captures. Was there any sort of consensus about which ones were critical versus helpful?

Barry Fishman: Hands down, in our study the screen captures were the most useful or helpful to teachers. I think this was because teachers were inventing their own uses for them, and the screen capture tool was flexible enough to accommodate different teachers’ ideas. For instance, some teachers would use them to have students document their progress, or to check that games assigned as homework were completed. Others would use them to prompt reflection, or give feedback on students’ interpretations of events in the game. This kind of flexibility speaks to the need for effective game features to support teachers’ own creativity and self-direction.

I think that dashboards would have been extremely helpful, if not for the technical difficulties our teachers encountered. I chalk this up to the lack of maturity in standards around how games report information, making it difficult to have dashboards that aggregate data across games. That’s an advantage closed ecosystems (like Manga High) have over more open environments.

Other issues had to do with the need to configure dashboards with student lists, and have students log in. Even though those are not high barriers, they are still barriers, especially for teachers without a lot of time or technical support.

gamesandlearning.org: Many of the case studies spoke of how helpful the “wrap-around” material was to using the game in the class. What are the key elements that teachers are looking for in that material?

Barry Fishman: When it comes to the materials that surround a game, it’s all about integration. How much information is there about how to use the game to meet curriculum goals? Are there other materials like videos or quiz questions that teachers can use to supplement game play? Are there graphic organizers to help teachers visualize the flow of the game? Good support materials are like a well-annotated teacher’s guide for a textbook. They help teachers make sense of the game and understand how to integrate it into their teaching. In our study, we again recognized that a portal like BrainPOP can be helpful here. Game designers and developers come from a broad range of backgrounds, with varying experience in K-12 teaching. BrainPOP was able to provide support materials that completed games.

gamesandlearning.org: Having taken a national snapshot through your survey and spent so much time with specific teachers in grades 5-8, do you see any major changes coming to the way teachers think about or use games? Will they remain small exercises that aim to engage students or could they be used more formally to gauge and assessment student abilities?

Barry Fishman: That’s a speculation question! But I do have a response. While we were conducting this study, organizations like GlassLab were embarking on ambitious game design projects tailored specifically for formative assessment of learning, such as SimCityEDU. The scope and timing of our study did not allow us to focus on games like that, but I perceive a general movement that combines games with data-driven tools for teaching, such as learning analytics, digital badges, and other approaches to personalized learning. These are all buzzwords at the moment, but we are starting to see a convergence that brings them together into useful and productive learning environments.

I’m bullish about the future of games for formative assessment, and for learning in general. But there are many pitfalls along the way. My hope is that our study points out areas where more attention results in features that make games more useful for more teachers.