Authors:
Felipe Sonntag Manzoni
1
;
Tayana Uchôa Conte
2
;
Milene Selbach Silveira
3
and
Simone Diniz Junqueira Barbosa
4
Affiliations:
1
Instituto de Computação, Universidade Federal do Amazonas, Manaus, Amazonas, Brazil, SIDIA Instituto de Ciência e Tecnologia, Validation Team, Manaus, Amazonas, Brazil
;
2
Instituto de Computação, Universidade Federal do Amazonas, Manaus, Amazonas, Brazil
;
3
Escola Politécnica, Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil
;
4
Departamento de Informática, Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, Rio de Janeiro, Brazil
Keyword(s):
Playability, Heuristic Sets, Playability Assessment, Game Assessment, Comparative Empirical Studies.
Abstract:
Background: Playability is the degree by which a player can learn, control, and understand a game. There are many and different Playability evaluation techniques that can evaluate different and numerous game aspects. However, there is a shortage of comparative studies between these proposed evaluation techniques. These comparative studies can assess whether the evaluated techniques can identify playability problems with a better cost-benefit ratio. Also, these studies can show game developers the evaluation power that a technique has in comparison to others. Aim: This paper aims to present and initially evaluate CustomCheck4Play, a configurable heuristic-based evaluation technique that can be suited for different game types and genres. We evaluate CustomCheck4Play, assessing its efficiency and effectiveness in order to verify if CustomCheck4Play performs better than the compared heuristic set. Method: We have conducted an empirical study comparing a known literature heuristic set an
d CustomCheck4Play. The study had 54 participants, who identified 49 unique problems in the evaluated game. Results: Our statistical results comparing both evaluation techniques have shown that there was a significant statistical difference between groups. Efficiency (p-value = 0.030) and effectiveness (p-value = 0.004) results represented a statistically significant difference in comparison to the literature heuristic set. Conclusions: Overall, statistical results have shown that CustomCheck4Play is a more cost-beneficial solution for the playability evaluation of digital games. Moreover, CustomCheck4Play was able to guide participants throughout the evaluation process better and showed signs that the customization succeeded in adapting the heuristic set to suit the evaluated game.
(More)