A Platform to Interest Young People in STEM using Robotics and AI

in a Playful Way

Fabian Gibert and Georg J. Schneider

Department of Computing, Trier University of Applied Sciences, Schneidershof, Trier, Germany

Keywords: e-Learning Hardware and Software, Children's Education using Computer Support and K12 Students,

Robotics.

Abstract: This paper describes an educational and affordable robotics design of hardware and software for a technical

platform for playful learning in STEM fields, providing a low entrance barrier and a smooth transition from

playing to developing, depending on the user’s age. The platform is installed in a toy car and operates the

vehicle motors. A smartphone app serves as remote control. In addition to the simple basic electrotechnical

structure for driving, which is comprehensible to users, the design can be extended with actuators or sensors

via standardized modules and interfaces. On the software side, experimentation possibilities arise to process

captured sensor values or to experiment with an AI-supported image recognition, allowing users to get an

insight into up-to-date discussions like AI-based image recognition int the context of autonomous driving. In

addition, the system offers smartphone app functions, such as an image display on the smartphone or

automated vehicle behavior. Manufacturer-independent and without proprietary specifications, this platform

opens up a flexible, expandable technical basis for playfully exploring the interaction of components in one's

own interest without prior knowledge or programming experience.

1 INTRODUCTION

Mathematics, computer science, natural sciences,

technology - almost no area of life today is not

influenced by the disciplines summarized in the

acronym “STEM”. It is becoming increasingly

important for future generations since this field is not

only in high demand from the labor market, but it also

affects our daily life. Knowledge in these areas is

essential for a country to be attractive as business

location. (U.S. department of education 2022) states:

“In an ever-changing, increasingly complex world,

it's more important than ever that our nation's youth

are prepared to bring knowledge and skills to solve

problems, make sense of information, and know how

to gather and evaluate evidence to make decisions.

These are the kinds of skills that students develop in

science, technology, engineering, and math,

including computer science—disciplines collectively

known as STEM/CS.“ Interesting young people in

technology should already start at an early age. Hence

an age-appropriate approach is needed and thus a

playful and experimental method must be provided to

gain experience. (Chatzopoulos et al., 2021) show in

https://orcid.org/0000-0001-7194-2394

their study with 10-11 years old students that the

integration of educational robots has a positive impact

on students’ acquisition of scientific concepts.

However the available platforms are rather childish or

expensive, when it comes to custom extensions for

individual projects (cf. section 2).

This paper presents the development of an open,

low-cost and modular technical platform that enables

an introduction in the field with low entry barriers.

The system shall spike the interest in current topics

such as robotics and AI through a playful approach.

Children and young people are thus given access to

explore modes of action and perspectives with a

platform that costs a total of 75€.

Toys always depict contemporary topics and

translate the world for young people into their world

of experience. The approach is quite different.

Accordingly, the approach presented here is one such

translation and deliberately represents a foundation of

interwoven different elements, much as the term

STEM suggests a scientific interweaving of multiple

directions.

Exemplary use of the platform in our example is

as a technology module for operating toy vehicles and

346

Gibert, F. and Schneider, G.

A Platform to Interest Young People in STEM using Robotics and AI in a Playful Way.

DOI: 10.5220/0011084900003182

In Proceedings of the 14th International Conference on Computer Supported Education (CSEDU 2022) - Volume 1, pages 346-352

ISBN: 978-989-758-562-3; ISSN: 2184-5026

robots. The realization as a multifunctional vehicle is

contemporary and introduces users to new aspects of

digitized mobility in a playful way. Especially a

modern user interface via a smartphone makes the

platform attractive to young adults and children. The

AI-based image recognition reflects the current

discussion about autonomous driving and gives a

glimpse of the challenges to the user. Especially it is

possible to train or replace this module. In a further

step, the platform can also be improved with

additional sensors and hence can learn to process

these inputs as well.

We see the platform as a teaser for children and

adolescents to start playing with a toy car using a

modern smartphone user interface, which reminds

users on race car games where they can tilt and yaw

the phone to control the car. Even young children

from 5 years on can start using the car as a simple toy.

Additionally, the vehicle provides several newly

features, as displaying the video image on the smart

phone but also recognizing images and reacting upon

them. Hence a broad spectrum of use cases is

available with a low entrance barrier into the field. It

starts from simple playing with the car over

exchanging parts of hard- and software provided by

more experienced users, which targets older children

from possibly 10 years upwards until developing

custom enhancements with hardware and/or software

components. This would be targeted to children of 13

years and older as well as adults.

2 RELATED WORK

The idea of a playful approach is not new, neither is

the teaching and explaining intention of systems to

make STEM topics tangible. Example from

university systems and commercial platforms will be

discussed in more detail in a further section.

As a general overview, (Sapounidis & Alimisis,

2020) give a good description of the current state of

the use of robots in education for young children.

2.1 Related Research

The PiBot system is a low-cost robotic platform with

camera for STEM education (Vega & Cañas, 2018).

The system has the same target group as our system.

The PiBot bases on a Raspberry Pie 3 controller with

sensors attached. The robot is rather targeted to

operate autonomously in a delimited environment,

whereas we want to emphasize the toy character by

providing a remote control on a smartphone.

Additionally, we want to integrate an AI component,

which is a hot topic today to showcase and

experiment with the capabilities of this technology, in

our case image recognition.

(Wang et al., 2019) describe a toy car with remote

control and image recognition. However, their system

strongly relates to the application domain.

Possibilities to user the car as an experimental kit for

technology acquisition is not intended.

(Chatzopoulos et. Al., 2020) present a low cost

robotic platform for building a robot with wheels

integrating sensors and actuators. They also integrate

a visual programming platform. In contrast to our

approach the robot is targeted to their educational

context. The robot does not look like a “real” car and

it does not integrate a camera and image recognition

using AI tools.

2.2 Related Commercial Systems

Lego has set itself the goal of “playful introduction to

STEM” (LEGO Group, 2021a) and offers solutions in

the form of the Lego Mindstorms series. However,

hardware and software are not technically freely

extensible or open-source based, so that applicability

and learning possibilities are always tied to the

manufacturer and the resources provided. As a

consequence, expensive components (LEGO Group,

2021b) also pose a significant financial threshold for

users. Despite the implementation of an independent

project on "PixyCam" (PixyCam, 2021), which takes

Lego compatibility into account, no Lego-supported

camera is available because the manufacturer does

not push this even after years.

The company DJI pursues a similar goal and offers

a learning robot as an “all-in-one solution” for

experimenting and programming called “DJI

RoboMaster Lernroboter” (DJI, 2021). This

comprehensive approach is provided for workshops

and learning processes though and addresses users with

training requirements. The resulting high costs as well

as the professional intentions and perspectives.

Therefore, they also form entry barriers for younger

people who are the target group of our system.

The need for programming and high prices keep

many potential young interested parties away from

corresponding offerings.

Our goal is that the educational approach we

follow, i.e. playful interaction as a starting point for

acquiring STEM knowledge and competencies need

not be limited to a small interested group of users.

Other children and young people can and should also

discover STEM content in a playful way. This work

demonstrates that it is possible to experience

comparable functionality without such high hurdles.

A Platform to Interest Young People in STEM using Robotics and AI in a Playful Way

347

Figure 1: System Architecture.

3 TECHNICAL CONCEPT

The solution presented can be conceptually divided

into two parts (see fig. 1). First, the platform consists

of hardware and software, which in the case of the

vehicle application is mounted directly on the vehicle

and provides connection and operation of sensors and

actuators. Second, a user interface is necessary for

interacting with the vehicle. The GUI is realized as an

Android app running on an off the shelf smartphone.

The smartphone, in turn, provides computational

capacity to implement, besides the GUI, an AI

component and provides further functionalities, such

as sensors, which can be used as additional input for

the GUI.

For the purpose of independence and scalability,

on the one hand a WiFi connection is used between

the platform and the app, whereby both are directly

connected and do not require any further network

technology or configuration like a WLAN router or

such. On the other hand, communication takes place

via the standardized HTTP, TCP and UDP protocols.

The images from the integrated camera as well as

sensor values are sent from the platform to the app

and displayed in the GUI. The app in turn sends

control or configuration commands to the platform.

This client/server architecture does not require an

internet connection. The network concept is not

Android or even smartphone specific and therefore

allows other input devices as clients.

The use of a microcontroller as the platform's

computing unit offers the option to connect and

operate different sensors and actuators. With a focus

on typical voltage levels and standardized electronic

components, the control concept allows additional

components to be used as actuators via a simple

connection. Bus interfaces enable several parallel

peripheral devices such as sensors. All hardware is

battery-powered, equivalent to conventional remote-

controlled vehicle models, which makes the vehicle

exchangeable too.

The GUI is designed as a remote control for

steering the vehicle with buttons for steering and

acceleration. Additional functions can be activated in

a menu. Likewise, parameters for operation can be

selected to individually configure the platform, i.e.

the car.

Through software modules that process data from

the hardware, a wide range of applications can be

developed. Some exemplary applications are already

implemented. As an example, the automatic

switching on of the lights is realized as soon as the

calibratable light sensor detects low brightness.

Furthermore, the automatic stopping of the vehicle is

implemented, as soon as the AI-based image analysis

detects a “Stop” sign that the vehicle is approaching.

Other models can be dynamically integrated into

the application. Depending on the training domain,

numerous other scenarios can be implemented as well

and users are able to explore either the use of AI-

components or the integration of a special behavior of

the car in a playful manner.

4 IMPLEMENTATION

The platform consists of a base of hardware

components. The microcontroller is programmed in

C++. It manages the access point, the communication

to the app and the control of the connected periphery

depending on the commands.

The Java app, designed according to MVP,

includes parts for communication, data processing,

graphical display for control, and image processing

CSEDU 2022 - 14th International Conference on Computer Supported Education

348

Figure 2: Java snippet for Tensorflow Integration.

through a machine learning model. This approach is

described in more detail in the next section.

Furthermore, the concept supports a modular and

extensible operation of the system to new applications

and offers the user new insights in the belonging

technologies.

4.1 Machine Learning in Android

The computing capacity of the smartphone allows the

use of a model trained on street signs

("SSDMobileNet"), the framework Tensorflow

(Tensorflow (Google LLC), 2022) realizes its offline

integration into the Java application, as fig. 2 shows

to some extent. The figure also shows the few simple

steps that must be performed within the program code

in order to perform this integration. A Tensorflow

interpreter instantiated with the model from the

resource folders receives the camera image for image

processing. This image was preprocessed for resizing

and transformed into a TensorImage. On this

database, the detection method, which allows

multiple detections per image, fills a hashmap with

the result data set. This is then evaluated and contains,

among other data, the location of the detection in the

image, the designation (“classes”) of the detected

object and a probability value (similarity of the

current image object to the training object). The

source code can be extended accordingly to integrate

further models. Hence there is a possibility for users

to experiment with AI-technology and have hands on

training and first-hand experiences. We believe that

this is very attractive for young people since the use

of AI in supported or autonomous driving is

constantly discussed in the media. As an additional

benefit, users get to know the limitations and

challenges using such a technology.

4.2 Modularization and Extensibility

In order to address the spectrum of problems that

MINT poses, the platform is modularly expandable in

various respects and therefore flexibly designed.

On the hardware side, the ESP32 microcontroller

(Espressif Systems, 2021) is extended by an expander

IC to provide more connection pins. It addresses, for

example, the motor control, an H-bridge to control the

DC motors. The expander IC also offers connection

pins for LEDs, for example to realize the front light

of the vehicle. But not only outputs but also inputs

can be defined to receive data. According to this

extension principle via analogue pins and level

control or via defined digital interfaces like SPI and

I2C, the platform can be extended by various sensors

and actuators. Fig. 3 shows an example of a circuit for

the SPI interface. The interfaces are inexpensive,

open source and often multilingual tutorials are

available. Therefore, they are used as key components

of our platform.

Figure 3: Example of multiple SPI slave devices to one

master device (SparkFun Electronics, 2022).

On the software side, the hardware extensions

described above, already result in numerous

possibilities that can be installed and activated in

order to extend the system. In addition, the app also

offers expansion potential, e.g. in adding further

graphical user interface elements. Parameterization is

already provided in the menu to configure the

behavior of the AI evaluation, user feedback such as

visual feedback or vibration alert, configuration of the

vehicle's steering, image compression, etc. according

to current usage intent. Accordingly, additional menu

items and functionalities can also be added.

Beyond the individual hardware and software

features, there are completely new ways in which

these enhancements can be used: by combining both

areas as well as adding further software modules for

A Platform to Interest Young People in STEM using Robotics and AI in a Playful Way

349

data processing such as the exemplary stop automatic,

the users can find their own paths and explore the

system playfully.

Figure 4: App usage to control the car via sensor.

Fig. 4 shows the app usage on the smartphone,

whose sensor detects the rotation and controls the

steering of the vehicle in the background. The live

image of the camera is visible in the center of the

smartphone.

5 CONCLUSIONS AND FUTURE

WORK

First, the results of this work will be summarized and

then discussed, and their further development will be

highlighted.

5.1 Conclusion

The technical platform, which finds concrete

application in the work as a modularly changeable

model car, is designed to be simple yet flexible and

thus expandable. It comprises a microprocessor, a

camera module, actuators and sensors and thus a

system of parameterizable input and output options.

Both the AI-based evaluation of camera images and

the programmable interaction of sensors and

hardware allow for the intended freedom of design

and experimentation. The technology is based on

available standard components.

This solution eliminates special, expensive

components or the need for users to individually write

software to make the system work. Especially

important for further extensions is that there are no

proprietary parts or vendor dependencies.

Based on this platform, we have realized a

modular architecture which opens new possibilities

for young users to discover and test digitalization and

mobility while playing and experimenting. They can

gain numerous insights from the STEM world within

their own horizon of knowledge and possibly

extending the horizon step by step. Having in mind

the different approaches or steps mentioned in the

beginning, the platform offers from simple playing

for younger children (5 years and older) over

changing preconfigured parts of hardware or software

modules for older children (10 years and older),

when the maturity and interest in the vehicle

hopefully grows to adolescents (13 and older), which

are even more advanced in their skills and want to

program the vehicle on their own or add completely

new behavior.

Experiences can not only be made only on the

positive possibilities of our technical environment,

but also the ambiguities and faults and its

recognizable limits, such as those of artificial

intelligence, which will be explained in greater detail

in the next section. Here, we see an interesting

starting point for older children and adults to gain

more insights into the field of artificial intelligence,

in our case image recognition in the context of

autonomous driving. Since this topic is discussed in

the media intensively nowadays, we se a potential

high interest to get experiences and more profound

knowledge in this field.

In contrast to the related systems presented in this

paper, this platform therefore offers young people a

low entrance barrier in the robotics and/or AI field. A

playful and independent way of getting started and

finding their way around has been created.

5.2 Future Work

The system has been initially developed having the

needs of a child of about 13 years in mind. The

potential user has been interviewed and integrated in

the development process as good as possible in the

period of the current travel and contact restrictions.

Further evaluation must be done in the future. Since

we target three different scenarios from “playing”

over “exchanging of parts” to “customizing the

vehicle” and additionally a forth possibly orthogonal

scenario, which relates to getting to know the AI we

have to develop different, partially long-running tests

in order to find out, if our hypotheses hold that there

car stipulates the interest in the underlying concepts

of the “toy” and leads to getting into the teams STEM

and AI.

The technical solution achieved can be further

developed on various levels. For example, better

images are possible with a higher-quality camera,

which, in addition to visualization, also optimizes

the connected image processing. Thus, more

CSEDU 2022 - 14th International Conference on Computer Supported Education

350

Figure 5: Correct interpretation ”Stop” Sign colors display red (left) and misinterpretation of ”Stop”sign as ”Do not enter“

(right).

sophisticated visual evaluation can be realized, but

also errors in detection can be reduced. Depending

on the environmental situation and the visual quality

of the replicated road signs in the example of the

vehicle application, the “Stop” sign was not correctly

detected in 100% of the cases, and the automatic stop

of the vehicle was therefore not always executed (see

fig. 5 and fig. 6).

These rare misinterpretations were caused, among

other things, by a too large distance to the object,

visual similarities between two signs, or resolution

limits of the camera images. Other objects in the

immediate environment however rarely interfered

with the correct recognition of the traffic signs. For

gaming purposes these success rates are probably

sufficient. Furthermore, misinterpretations are real

existing phenomena when dealing with the

corresponding technology. Their occurrence in the

scenario is thus quite an expected representation of

the real world and should not be interpreted

exclusively as a fault or shortcoming of this solution.

In accordance with the numerous possible

variants of use of the platform, possible technical

compositions of the system must be built up in the

future and examined for any errors and

incompatibilities. Existing functions must be

optimized, and additional modules should be

included.

Further automatic functions beneath the presented

“Stop” function using AI-based image recognition

and the automatic light switching by the brightness

sensor can be added as well in order to provide an

ever-richer stock of settings with which young users

can research. These examples are plausible reactions

within the domain and display an exemplarily

implementation of such a behavior.

Equally helpful for user acceptance can be a

visualization of the components involved or the

technical conditions and limitations. Complementary

explanations of STEM aspects in the app can also

accompany or suggest a setup.

In addition to various interchangeable,

compatible machine learning models, Tensorflow,

among others, also provides training of models for

new domains and custom data with tools and

guidance. In this direction, the platform benefits from

the option to integrate the seemingly limitless

possibilities of AI usage into the app via the

framework.

A next extension can be a multi-user operation. If

currently only a single user can act with his

smartphone per platform, several users are

conceivable in the future, who complement or

challenge each other playfully.

The example of the vehicle application illustrates

that more than one participating platform can be used

as a technical module. For example, two vehicles

could also be designed with the goal of reacting to

each other based only on their technical capabilities.

In the area of GUI design, new requirements arise

as more components and scenarios are realized. As a

consequence, an almost completely configurable

design with visual elements and functions is

conceivable, which considers or simplifies as many

use cases as possible.

REFERENCES

Chatzopoulos, A., Kalogiannakis, M., Papadakis, S.,

Papoutsidakis, M., Elza, D., & Psycharis, S. (2021).

DuBot: An Open Source, Low Cost Robot for STEM

and Educational Robotics, in: Handbook of Research

on Using Educational Robotics to Facilitate Student

Learning (pp. 441 - 465), IGI Global

Chatzopoulos, A., Papoutsidakis, M., Kalogiannakis, M. &

Psycharis, S. (2020) Innovative Robot for Educational

Robotics and STEM, Intelligent Tutoring Systems, in:

International Conference on Intelligent Tutoring

Systems (ITS 2020), Springer International Publishing,

(pp. 95-104)

DJI, Product site (2021) „Robomaster EP Core“,

https://www.dji.com/de/robomaster-ep-core, download

12/3/2021

A Platform to Interest Young People in STEM using Robotics and AI in a Playful Way

351

Espressif Systems, (2021), Espressif Modules „ESP32-S

Series“, https://www.espressif.com/en/products/modu

les, 2021, download 11/3/2021

LEGO Group (2021a) „Lego Interessen: Lernen MINT mit

Lego Spielzeugen“, https://www.lego.com/de-de/

categories/stem, download 11/20/2021

LEGO Group (2021b) „Lego Home: Mindstorms EV3“,

https://www.lego.com/de-de/product/ev3-intelligent-

brick-45500, download 11/20/2021

PixyCam, (2021) „Introducing Pixy2 for Lego Mindstorms

EV3“, https://pixycam.com/pixy2-lego/, download

5/17/2021

Santos, R. (2019) „ESP32 I2C Communication“,

https://randomnerdtutorials.com/esp32-i2c-communica

tion-arduino-ide/, download 1/26/2022

Sapounidis, T. & Alimisis, Dimitris. (2020), Educational

robotics for STEM: A review of technologies and some

educational considerations, in: Science and

Mathematics Education for 21st Century Citizens:

Challenges and Ways Forward (pp.167-190), Nova

science publishers: Hauppauge, NY, USA,

SparkFun Electronics, (2022), „Serial Peripheral Interface

(SPI)“, https://learn.sparkfun.com/tutorials/serial-

peripheral-interface-spi/all, download 1/26/2022

Tensorflow (Google LLC) (2022), „Why TensorFlow“,

https://www.tensorflow.org/about?hl=en, download

1/26/2022

U.S. department of education (2022) download 1/27/2022

https://www.ed.gov/stem

Vega, J. & Cañas, J. M. (2018), PiBot: An Open Low-Cost

Robotic Platform with Camera for STEM

Education in: Electronics 2018, 7(12), 430;

https://doi.org/10.3390/electronics7120430

Wang, P, Tian, J, Niu, H, & Chen, Y. (2019) "Smart

Agricultural In-Field Service Robot: From Toy to

Tool." Volume 9: 15th IEEE/ASME International

Conference on Mechatronic and Embedded Systems

and Applications. Anaheim, California, USA.

August 18–21, 2019. V009T12A050. ASME.

https://doi.org/10.1115/DETC2019-97497

CSEDU 2022 - 14th International Conference on Computer Supported Education

352