Research on User-Defined Game Generation Based on AI

Chengyu Ma

Institute of Problem Solving, Shanxi University, No. 63, South Zhonghuan East Street,

Xiaodian District, Taiyuan, Shanxi, China

Keywords: Artificial Intelligence, Assisted Game, Convolutional Neural Network, Reinforcement Learning, Adversarial

Network.

Abstract: In today's digital age, the traditional game production process usually requires professional game developers

to invest a lot of time and energy, and it is difficult for ordinary users to participate in game creation. This

article reviews the existing results of game design using artificial intelligence technology, aiming to lower the

threshold of game development for the development of automatic game generation systems for ordinary users,

improve development efficiency, enrich players' gaming experience, and promote innovation and

development in the game industry. At present, three types of methods have been summarized, namely,

methods based on reinforcement learning, methods based on generative adversarial networks, and methods

based on convolutional neural networks to achieve auxiliary game design. This study is conducive to

promoting the formation of a new entertainment ecology in which all people can participate in creation. This

research topic takes user needs as the starting point and integrates artificial intelligence related technologies.

It is highly innovative and practical.

1 INTRODUCTION

Since the rise of artificial intelligence technology,

games have always been one of its important

application scenarios. Recently, at the 2024 Game

Developers Conference (GDC) conference, many

domestic and foreign manufacturers focused on

discussing the application of artificial intelligence in

the game development process. (Game Developers

Conference, 2020) The rapid advancement of

artificial intelligence technology has greatly

influenced and transformed all aspects of game

development. From content generation, design

optimization, to player behavior analysis and

personalized experience, artificial intelligence has

become a key tool for modern game development.

The GDC conference showcased the latest AI-driven

innovations and explored how to use technologies

such as machine learning to automate and enhance

game development. AI-driven player modeling and

adaptive mechanisms make it possible to create more

attractive gaming experiences. As the game industry

continues to develop, artificial intelligence has shown

https://orcid.org/0009-0000-3471-9556

great potential in changing the way games are

conceived, developed, and experienced, and will

surely shape the future of interactive entertainment.

Driven by the diversification of content needs and the

optimization of development efficiency, AI-assisted

game design is widely anticipated and is also a hot

area of academic attention. Game software

development is an important research direction in the

field of game development, aiming to use computer

algorithms to automatically generate levels,

characters, storylines and other content in the game to

improve the playability and creativity of the game.

Game artificial intelligence has always been an

indispensable and important part of computer game

development. AI characters can effectively enhance

the player's gaming experience by supporting the

player's pause state and dynamically managing

dramatic environments. In addition, AI designers use

procedural content generation technology to support

and enhance the development of individual games.

(Riedl, Zook, 2013) In the future, it is envisioned that

AI production engineers will take on more new roles,

not only enhancing and expanding the game

production process, but also supporting the entire

122

Ma, C.

Research on User-Deﬁned Game Generation Based on AI.

DOI: 10.5220/0013234800004558

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st International Conference on Modern Logistics and Supply Chain Management (MLSCM 2024), pages 122-128

ISBN: 978-989-758-738-2

real-time operation process in the game, promoting

cross-game interoperability, cultivating a strong

player community, and realizing the integration of

real and virtual environments. The realization of this

vision is due to the large amount of data resources

accumulated by the game development industry. By

integrating AI technology, game development will

move to a new stage, bringing players a more

immersive and personalized gaming experience.

In addition, AI technology is also promoting the

democratization of 3D model creation, especially in

the field of VR games. Tools like Google Genie

enable creators to generate game content through

prompts and sketches, although converting 2D to 3D

still faces challenges in data sets and computing

requirements. Artificial intelligence can reduce the

motivation of game development, stimulate

innovation, and simplify content creation, allowing

more creators to realize their vision of the virtual

world. As technology develops, user-generated VR

experiences will become more convenient, and future

VR games will enhance diversity and creativity

(Begemann, Hutson, 2024).

This article will explore the various applications

of AI in game production and look forward to its

future development trends.

This article introduces three different types of AI-

assisted game design methods: reinforcement

learning-based methods, generative adversarial

network-based methods, and convolutional neural

network-based methods.

In the field of game plot generation, researchers

have proposed several innovative methods based on

machine learning. First, the reinforcement learning-

based method allows AI agents to learn to generate

the best game plots through interaction with the

environment, and optimize the generation quality

based on player feedback to provide players with a

personalized gaming experience. Second, the method

of using generative adversarial networks (GANs) can

produce high-quality, realistic game content, break

through the limitations of rule-based games, and

generate a unique gaming environment. Third,

convolutional neural networks, with their powerful

visual perception capabilities, can help AI agents

understand and analyze game scenes, provide

important contextual basis for plot generation, and

combine reinforcement learning to learn the optimal

decision-making strategy. These innovative machine

learning-based methods have opened up new

possibilities in the field of game plot generation.

From the perspective of difficulty and

implementation, there are still some differences in

these three directions. (Andrew, James, 2024) The

first and second are relatively more feasible, while the

third may require more technical breakthroughs.

2 GAME LEVEL

RECOMMENDATION

METHOD AND DEVICE BASED

ON REINFORCEMENT

LEARNING

Reinforcement learning is an innovative method of

machine learning that allows intelligent agents to

complete tasks through interactive learning with the

environment. The agent first observes the state of the

environment, selects and executes actions, and

obtains corresponding reward feedback. Although

there is no pre-labeled training data and no evaluation

of each action, the agent can learn from the lagged

overall feedback and ultimately obtain the optimal

decision-making strategy to obtain the maximum

cumulative reward. This learning model is suitable

for many application scenarios that require

autonomous learning, such as autonomous driving

and robot control, and provides new possibilities for

the development of these fields.

The goal of this chapter is to give a case study of

game reinforcement learning applications and to

explore in depth the working mechanism and failure

reasons of reinforcement learning algorithms in

practice. Although these algorithms can converge to

the optimal strategy under ideal conditions in theory,

these ideal conditions are often difficult to meet in

complex game environments, so it is necessary to

further study the performance of reinforcement

learning in practical applications.

In addition to the algorithm itself, other factors

such as the choice of representation, the encoding of

domain knowledge, and heuristic methods also have

an important impact on the application effect of

reinforcement learning in games.

This example provides a game level

recommendation device and method based on

reinforcement learning as shown in Figure 1. The

device includes:

A state input unit, which collects game-related

data of the player.

A first neural network, which recommends game

levels of matching difficulty to the player based on

the game-related data collected by the state input unit.

A second neural network, which generates

evaluation information based on the game-related

Research on User-Deﬁned Game Generation Based on AI

123

data collected by the state input unit and the game

levels recommended by the first neural network.

And a parameter updating unit, which updates the

first parameter of the first neural network and the

second parameter of the second neural network based

on the evaluation information. Through the

embodiment of the present application, game levels

of appropriate difficulty are adaptively recommended

to players (Zhu, 2023).

Figure 1: Device diagram (Picture credit: Original).

Therefore, adaptive deep reinforcement learning

is used to dynamically and real-time adjust and match

the game difficulty according to the player's skill

level and game status, so that the player's skill level

and the difficulty of the game level are fully matched,

allowing the player to obtain a satisfactory gaming

experience.

3 ANIME-STYLE GAME

BACKGROUND GENERATION

METHOD AND PLATFORM

BASED ON GENERATIVE

ADVERSARIAL NETWORK

Next, the paper will give a specific example involving

the field of robot control, and more specifically, a

method and platform for generating anime-style game

backgrounds based on generative adversarial

networks.

3.1 Specific Steps

First, a dataset of game background images is

constructed. Each scene contains images at three

different time nodes, and each image is labeled with

attributes. Next, a color transfer network for

generating color segmentation maps and a style

reconstruction network for reconstructing anime style

maps are constructed. The pre-trained Visual

Geometry Group (VGG) network and Gram matrix

loss function are used to optimize the structure and

style of the generated images. In the model

preparation stage, Python is used to create a

generative adversarial network for anime style

transfer. The edge detection and image segmentation

algorithms are used in the input stage to generate line

drawings and color block drawings of photos, which

are combined with anime style pictures for

unsupervised training. Then, a conditional generative

adversarial network is used to perform color transfer

on the color segmentation map of the photo. The color

block map is combined with the time node label and

input into the generator to generate images at

different time nodes. Finally, the edge extraction and

color segmentation algorithms are used to process the

training pictures, and the network models of the color

transfer and style reconstruction stages are

established. The photos and time node labels are input

into the generator to generate the target anime style

game background images (Liu, Wang, Zhu, 2022).

Generators G1 and G2 are composed of two parts:

encoding and decoding. The encoding part includes a

fully connected layer, a downsampling layer, and

multiple residual blocks, and the decoding part

includes an upsampling layer and a fully connected

layer. Each time the sample is downsampled, the

feature map size is halved, and each time the sample

is upsampled, the feature map size is doubled.

G1 is responsible for converting the color

segmentation map of the photo into the color features

of a specific time node. The photo and time tag are

input into G1, and the color segmentation map that

conforms to the time semantics is output.

Discriminator D1 determines whether the input is an

anime color segmentation map.

G2 is responsible for encoding and decoding the

line map and the color segmentation map into a

complete anime-style picture. The line map of edge

detection and the color segmentation map output by

G1 are spliced and input into G2, and the target

anime-style background is output. Discriminator D2

determines whether the input is an anime picture.

The framework is implemented through two

generative adversarial networks. G1 and D1 are

trained first, and then G2 and D2 are trained, and

finally an anime-style game background is generated.

MLSCM 2024 - International Conference on Modern Logistics and Supply Chain Management

124

3.2 Analysis

This method uses the powerful ability of generative

adversarial networks to convert the color and style

features of images into anime styles, providing a new

solution for game background generation. The

generator uses the encoding-decoding structure to

fully explore the potential features of the image, and

the discriminator ensures that the output results meet

the requirements of the anime style. This end-to-end

learning method saves a lot of manual design and

debugging, and can greatly improve the efficiency of

game background generation. However, in practical

applications, some key technical problems still need

to be solved, such as how to further improve the

authenticity and diversity of the generated effects,

and how to achieve the deep integration of automatic

generation and manual creation.

Table 1 below is a comparison table of the

generation effects of this example and various

methods. Each group of rows from top to bottom are

the outputs under the daytime, dusk, and night labels.

The lower right corner of each image in the output

result of NST is the selected style image. When the

style map is selected appropriately, good results can

be output, but in practical applications it is difficult to

ensure that each photo input has a suitable style map.

The output result of CycleGAN has the color of anime

pictures, but the texture is too complex, and it is

impossible to generate images with different time

features. MUNIT can generate style images with

different time features, but the images generated by

MUNIT have largely lost the original image structure.

Based on CartoonGAN, the paper adds temporal

condition input and name it CartoonGAN. Its output

image has temporal features, but the color is dim and

the texture is not smooth enough.

Table 1: Comparison results (Liu, Wang, Zhu, 2022).

Way

NST

Cycle

GAN

MUNIT

artoo

GAN

ECartoon

Result

Cartoon

Style

234.32 154.16 144.37 117.22 80.2 72.48

Cartoon

Daytime

276.41 212.37 217.95 163.97 158.50 143.87

Cartoon

Dusk

255.35 223.7 214.06 171.39 159.32 145.19

Cartoon

ight

238.69 209.27 215.95 167.48 162.53 153.69

3.3 Result

This example outputs colors with different temporal

features and smooth textures. The lines of the fences

and branches on the road become straight, and the

distant view is filled with a uniform atmosphere color,

which is a technique often used in game original

painting. In order to make the evaluation more

objective, the FID distance indicator is used to

evaluate the quality of the generated image. FID is an

indicator used to evaluate the effect of image

processing. It uses a pre-trained ImageNet model to

extract high-level features of the image and calculate

the distance between the two image domains. When

calculating FID for anime image domains with

different temporal features, CycleGAN and

CartoonGAN each use the same output domain, while

other methods use the output of each group of specific

time labels and the corresponding anime time feature

domain for calculation. Therefore, this example not

only achieves the best results in the anime style FID

indicator, but also achieves the best results in the

comparison of FID indicators in each temporal

feature domain.

4 DESIGN OF AI FOR THE DARK

SOULS GAME BASED ON

CONVOLUTIONAL NEURAL

NETWORK

Game developers are trying to apply artificial

intelligence and machine learning to games to

improve the behavioral logic of game characters and

the adaptability of the game difficulty curve, so that

the behavior of virtual characters is closer to that of

intelligent creatures, thus enhancing the player's

sense of involvement and achievement.

In terms of game screen data training, this

example uses an improved convolutional neural

network model, such as AlexNet, to achieve efficient

and accurate model training effects. Taking "Dark

Soul" as an example, it shows how to use deep

learning technology to enable game characters to

achieve automated and adaptive attack and defense,

and create virtual characters that are closer to human

behavior. Through artificial intelligence and machine

learning technology, game developers are working

hard to improve the intelligence level of game

characters and enhance players' gaming experience.

This is a vibrant research direction (

Kusiak, 2019).

Through methods such as Canny edge detection

algorithm and color space compression, the numerical

processing of the game UI is realized, and Grab Cut

technology is used to extract the main body of the

image, accumulating a large amount of data for

Research on User-Deﬁned Game Generation Based on AI

125

subsequent model training. Based on deep learning,

the AlexNet model was selected as the basis, and

through hyperparameter tuning, the error function of

the model was significantly reduced and the

performance was improved. The trained program was

applied to the actual game environment for testing,

which verified that the proposed game AI can

simulate some of the player's behaviors in the game.

This research makes full use of computer vision

and deep learning technology to achieve

preprocessing and efficient modeling of game UI

data, providing strong support for the intelligence of

game characters. Through verification in actual

games, the effectiveness of the proposed method is

also proved. This idea of applying artificial

intelligence to game development helps to promote

the intelligence of game character behavior and the

optimization of game experience.

4.1 in Terms of Image Preprocessing

The following image preprocessing measures provide

a high-quality data foundation for subsequent model

training.

Using the Canny edge detection algorithm, blood

bar information is extracted from the game UI, and

the character blood volume value is obtained by

analyzing the edge ratio. The Canny algorithm

includes Gaussian filtering, gradient calculation,

weak edge suppression, and hysteresis processing.

In order to reduce the computational complexity,

the RGB color space is converted to the grayscale

space A, and the image subject is highlighted by

increasing the contrast. Specifically, the formula

A* = A ± 1.3σ is used for transformation.

The GrabCut algorithm is used to extract the

image subject. The algorithm uses image texture and

boundary information to achieve interactive image

segmentation through the minimum cut algorithm.

4.2 in Terms of Model Training

This study chose to use the Adam optimization

algorithm. Adam is a stochastic optimization method

based on adaptive learning rate. It has the advantages

of simple implementation, high computational

efficiency, and low memory requirements. It is very

suitable for processing large-scale data and parameter

optimization problems. Adam calculates the adaptive

learning rate of different parameters by estimating the

first-order and second-order moments of the gradient.

It has the advantage of convergence speed in the

online optimization framework, and its effect is better

than other stochastic optimization methods in

practice. Based on these advantages, this study chose

to use the Adam optimization algorithm when

optimizing the model loss function. The specific

implementation details of Adam can be found in

relevant literature and will not be repeated here

(Wang,2022).

In general, the use of an efficient Adam optimizer

in the model training stage helps to improve training

performance and accelerate model convergence. This

lays the foundation for the efficient training of game

AI systems.

Implementation under the Tensor flow framework

This paper simplifies the behavior of game characters

into four types as shown in Table 2, which is the

output of this set of vectors after the grayscale matrix

is input into the entire convolutional neural network.

Then select the Adam optimizer for training.

Select 100 batches in 20 epochs for iteration. It can

be seen that as the number of training times decreases,

the value of Loss Function continues to decrease. The

Loss Function selected here is the cross entropy of the

entire classification. This value can reflect the

difference information between two probability

distributions in Shannon information theory.

Table 2: Game character behavior.

ame The

correspondi

ng vecto

The

corresponding

utton

Action

Attack

{1,0,0,0} Left key

Cause damage to

non-defensive

enemies

Plus

shield

defense

{0,1,0,0} Right key

Avoid opponent

damage when you

ave enou

h ener

Roll to

avoid

{0,0,1,0} Space

Avoid enemy

damage and

distance

Drink

element

ottle

{1,0,0,1} R Reply HP

4.3 Actual test results of the model

During the game, Researchers can continuously take

screenshots and let the previously trained model make

predictions to give the operations that should be

performed at this time. Researchers can also use

Python programming to simulate the corresponding

key presses on the keyboard to test the actual effect

of the game AI here.

Hold the shield to defend and roll backwards

when the enemy attacks. As shown in Figure 2.

MLSCM 2024 - International Conference on Modern Logistics and Supply Chain Management

126

Figure 2: Attack game action (Wang, 2022).

It can be found that when the model captures the

enemy in the picture and shows an attacking tendency,

it allows the player to adopt a defensive or backward

rolling evasion strategy with a certain probability.

Active attack when the enemy is in a non-defense

state, it can be found that when the model captures the

enemy in the picture, it will actively attack the enemy

in a non-defense state. As shown in Figure 3.

Figure 3: Non-defensive game moves (Wang, 2022).

5 CHALLENGES AND

PROSPECTS

In fact, AI+games are facing some challenges, but

their future development prospects still exist.

First, at the technical level, the application of new

technologies such as AI needs further breakthroughs

and maturity, and its application in game production,

interaction, etc. needs further exploration and

verification. At the same time, fluctuations in policy

and regulatory environment may also affect the

development of AI in the game field. In addition,

there is uncertainty as to whether the recovery process

of the game accommodation end continues and

whether the application of AI technology can

continue to keep up with the changing market

demand.

But on the other hand, with the continuous

breakthroughs of AI technology in content

production, personalized experience, etc., AI+games

will become an important driving force for the

development of the industry. Under the general

environment of policy support and revenue recovery,

the application of AI in game production, operation

and other links will be further deepened, bringing

players a more immersive and interactive game

experience. In the next five years, Shanghai Pudong

New Area will invest more than 10 billion yuan to

support content research and development and

technological innovation in the game industry, which

will inevitably drive AI+games. Overall, although

AI+games are still facing some challenges, their

development prospects are still valid. The dual-wheel

drive of technological innovation and policy support

will promote the wider and deeper application of AI

in the field of game realization, bringing new

innovation and development momentum to the game

industry.

Overall, with favorable policies and continuous

improvement in the input side, the deep integration of

artificial intelligence and games will surely promote

exciting innovation and development in the game

industry. Game companies can further explore the

potential of artificial intelligence in game production

and operation, bring players a more immersive and

interactive gaming experience, and promote the high-

quality development of the entire industry. Overall,

with favorable policies and continuous improvement

in the input side, the deep integration of artificial

intelligence and games will surely promote exciting

innovation and development in the game industry.

Game companies can further explore the potential of

artificial intelligence in game production and

operation, bring players a more immersive and

interactive gaming experience, and promote the high-

quality development of the entire industry. (CGIGC,

2024)

The application of AI in games is still in its

infancy, and many legal, ethical and technical

obstacles need to be resolved. At present, the legal

ownership and copyright protection of games that use

AI to generate assets are unclear, which makes it

difficult for existing intellectual property owners to

use third-party AI models in their production

processes. However, the opportunity lies not only in

making existing games faster and cheaper, but also in

creating new AI games. With systems like generative

agents, personalization, AI narratives, dynamic world

building, and AI co-pilots, the paper may be on the

verge of seeing the first “ never-ending” game

created by an AI developer.

6 CONCLUSION

In summary, this article reviews three existing

methods for game-assisted design using artificial

intelligence technology: methods based on

Research on User-Deﬁned Game Generation Based on AI

127

reinforcement learning, generative adversarial

networks, and convolutional neural networks. These

technologies help to automatically generate game

content and provide a more user-friendly game

creation environment for ordinary users.

Artificial intelligence technology has made great

progress in the field of games. Starting from the

success of reinforcement learning algorithms in the

classic game TD-Gammon, AI technology has been

widely used in various games, including video games,

real-time strategy games, first-person shooters, and

role-playing games. Although the success of TD-

Gammon cannot be repeated in every game, there are

still many promising results and valuable lessons to

learn.

In terms of image generation, deep learning-based

methods show great potential. With the

popularization of the Internet, users can easily

generate game backgrounds through a browser. This

method can effectively convert the input image into

anime-style game background images and differential

images under different time conditions while

maintaining clear edges. At the same time, the

friendly user interaction interface and the web

platform based on familiar interaction logic greatly

improve the user experience.

Convolutional neural network is also an effective

game AI design method. The game UI is processed

by edge detection techniques such as Canny operator,

image memory is reduced, and a convolutional neural

network model based on AlexNet is constructed to

simulate player behavior. Although this method

performs well in actual game environment tests, it

may have certain limitations in processing sequence

data and dynamic decision-making and model

interpretability compared to reinforcement learning

and generative adversarial networks.

Generative adversarial networks are good at

learning from existing sample data and generating

new game materials, which improves the speed and

accuracy of training. However, this method is

difficult to train, prone to problems such as mode

collapse, and the generated content may be

unreasonable, and the computing resources required

are also high.

In actual game design, multiple AI technologies

are usually combined to give full play to their

respective advantages to achieve better results and

experience. In general, the use of artificial

intelligence for game concept design and iterative

optimization has injected new vitality into the game

industry and provided valuable opportunities for the

creative process.

REFERENCES

GDC (Game Developers Conference). 2024.8.20.

https://gdconf.com/event-details

Riedl, M. O., & Zook, A., 2013. Artificial Intelligence for

Game Production. 2013 IEEE Conference on

Computational Intelligence and Games (CIG), Niagara

Falls, ON, Canada, 1-8.

doi:10.1109/CIG.2013.6633663

Begemann, A., & Hutson, J., 2024. Insights on AI-Assisted

Game Development: A Case Study on Integrating

Generative AI Tools into Creative Workflows.

Metaverse, 5(2), 2568.

Kusiak, A., 2019. Convolutional and Generative

Adversarial Networks in Manufacturing. International

Journal of Production Research, 58(5), 1594–1604.

https://doi.org/10.1080/00207543.2019.1662133

Youyou Interactive (Beijing) Technology Co., Ltd., 2023.

Method and Apparatus for Game Level

Recommendation Based on Reinforcement Learning:

CN115659054B.https://wwwv3.cqvip.com/doc/patent/

3186975461

Gutiérrez-Sánchez, P., Gómez-Martín, M. A., González-

Calero, P. A., & Gómez-Martín, P. P., 2021.

Reinforcement Learning Method for Evaluating the

Impact of AI Changes on Game Design.

Liu, Q., Wang, Y. M., Zhu, F., 2022. A Method and

Platform for Generating Anime-Style Game

Backgrounds Based on Generative Adversarial

Networks: CN114917583A.

Jais, I. K. M., Ismail, A. R., & Nisa, S. Q., 2019. Adam

Optimization Algorithm for Deep Neural Networks.

Journal of Engineering. Data Science, 2(1), 41-46.

Wang, J. 2022. AI Design for the Game "Dark Souls" Based

on Matrix Neural Networks. Digital Communication

World, (02), 184-186. DOI:10.3969/J.ISSN.1672-

7274.2022.02.062

China Game Industry Report (CGIGC). 2024.8.20.

https://www.cgigc.com.cn/report.html

MLSCM 2024 - International Conference on Modern Logistics and Supply Chain Management

128