"The widespread interest in the learning and motivation benefits of serious video games has not been balanced by a robust discussion about evidence for their pedagogical effectiveness.... As of now, the evidence is solidly against the proposition that [serious] games will replace direct instruction," asserts noted academic Richard E. Clark in his article in the May/June 2007 issue of Educational Technology (pp. 56-59).
Clark's piece also quotes B. Sawyer in addressing entrepreneurs: "We still are dealing with a huge research gap in serious games, but so far it hasn't hurt things because people are still getting new projects online. At some point, however, the justification and design issues related to determining the return on investment and outcomes from game-based approaches may become too hard to overcome without more and better research."
While I disagree with Clark that the question is whether serious games are going to replace classrooom instruction, my own recent review of the literature supports these concerns about evaluation. In fact, although there is more and more ink being devoted to information about serious games in academic publications and market analysis reports, there is a paucity, if not absence, of literature reflecting the results of empirical research.
"But why," one might ask, "must I bother with evaluation if I'm already doing assessment? After all, if learners/players learn what they are supposed to, the design/development of the serious game is obviously good."
(Read this if you wondering about the difference between assessment and evaluation.)
Program or product evaluation seeks to answer comprehensive questions about the effectiveness of the product in addressing a stated problem. These include:
- Do learners accomplish the learning objectives, reliably and repeatably? Do they do so better or less well than if another instructional medium had been used?
- Do learners transfer new learnings from practice to performance environment? Do they do so better or less well than if another instructional medium had been used?
- Are there positive cost-benefit ratios associated with the serious game? How do those ratios compare if another instructional medium had been used?
The astute reader will observe that these questions go to business issues just as much as to pedagogical concerns. Without being too cynicial, every business manager ultimately is concerned with just one question, the last bullet point: is this the best way for me to spend my money on this problem?
Precious few of us have real answers to this question.
Now is the time for academic and other research organizations to put the serious questions to the test. In order for them to do so, they need projects-in-development from the practitioner/developer community. The interest in forming these partnerships should come from both sides. And funding must be allocated to their efforts to ensure valid, reliable, and robust research studies are conducted and analyzed.
Sawyer's prediction came in 2005. The day of reckoning is coming for serious games. Will we be able to prove our worth to the budget-keepers?
I'm serious,
Anne
