8
DISCUSSION
In this section, we discuss four things based on the
results of the main experiment. First, we discuss the
impact of implementing confusion in game agents. In
the result of the main experiment, [DQN, Imp, Con]
was able to acquire more human-like behavior than
[DQN, Imp] in contrast to little declining playing skill.
It was shown that [DQN, Imp, Con] was able to
obtain more human-like behavior than the method of
the previous research. In addition, in the free
comments about [DQN, None] and [DQN, Imp],
there were many negative opinions about human-like
behavior such as easily defeating enemies in
situations where Mario was surrounded by enemies.
By contrast, In the free comments about [DQN, Imp,
Con], there was no such opinion. We consider that our
method solved the problem of a game agent that
unnatural mechanical behavior occurred in scenes
where Mario was surrounded by enemies and blocks.
Next, we discuss whether the biological
constraints introduced into DQN were sufficient. In
the result, there was still a gap in evaluation between
[DQN, Imp, Con] and [Player]. Since DQN without
biological constraints has a high playing skill, we
consider that even if we add more biological
constraints, we can acquire human-like behavior
without lowering playing skill much. In our proposed
method, “confusion” solves the problem on the
condition such as when the enemy is dense. There are
still many biological constraints on such specific
conditions. We consider that implementing them as
much as possible will solve the mechanical behavior
problem and the game agent can look like an almost
human play.
We discuss in what situations the game agent
obtained by this proposed method would be useful. In
recent years, game agents for NPCs for competitive
games have been trained by using a large amount of
battle data, but the cost of development is huge. By
using our proposed method, it is possible to
automatically acquire a game agent that shows
human-like behavior and that players can enjoy. As
for other usage, in recent years, there are many people
who enjoy watching game play videos. It is thought
that it is possible to create a game agent that is fun for
those people to watch as entertainment.
Finally, we discuss the experimental method. In
our paper, human-likeness was verified by subjective
evaluation by participants. However, in such a
subjective evaluation experiment, the result may vary
greatly depending on participants because the
participants’ feeling is different individually, and the
persuasiveness of the results may be relatively low.
To solve this problem, it is necessary to quantitatively
define human-likeness and conduct experiments.
However, human-likeness is ambiguous and difficult
to measure quantitatively. Almost all previous studies
only employ subjective evaluation experiments like
our research. If human-likeness is defined
quantitatively, the ambiguity of evaluation will
disappear, and it will make a great contribution in the
field of human-like agent generation.
9
CONCLUSIONS AND FUTURE
WORK
In this paper, we combined DQN with biological
constraints to realize a game agent with human-like
behavior. The purpose of our research is to
automatically generate a game agent that have a game
skill of an advanced player and perform human-like
behavior. From the result of the preliminary
experiment, the DQN-based agent obtained more
mechanical behavior than the agent based on Q-
learning when only the biological constraints
“fluctuation”, “delay”, and “tired” were introduced.
To solve the problem, we proposed a new biological
constraint “confusion”. As a result of the subjective
evaluation experiment, it was shown that the game
agent that introduced our proposed method performed
more human-like behavior than the method of
previous research.
As our future work, we need to consider two
points:
• Explore new biological constraints;
• Apply our method to other game genres.
Since a game agent with biological constraints learns
to perform the optimal actions within those
constraints, we consider that more complete human-
like behavior can be acquired with high game skills
by introducing additional biological constraints. An
example of a biological constraint that we are
considering is “carelessness”. It is a psychological
constraint that makes people act without thinking
deeply when things are going smoothly. We should
verify whether human-like behavior can be obtained
by adding such new biological constraints.
In addition, one of the purposes of this proposed
method is that it can be applied to any game genres,
like the research by Fujii et al. Therefore, it is
necessary to apply it to games other than Infinite
Mario Bros. and verify whether it can perform
human-like behavior sufficiently. However, the
problem is that, in games with little movement such
as simulation games, it is not possible to visualize
sequential actions as in action games or shooting