Assistance System for the Interactive Machine Adjustment

(of a Tufting Machine)

Dominik Huesener and Jürgen Rossmann

Institute of Man-Machine-Interaction, RWTH Aachen University, Ahornstrasse 55, 52074 Aachen, Germany

Keywords: Digital Twin, Assistance System, Simulation, Textile Engineering.

Abstract: The paper illustrates how a digital twin, a virtual representation of a physical asset, and simulation software

can be used to find best operating parameters and guide an operator through the adjustment process of the

machine. The user interacts with the digital twin either through a 3D GUI or using augmented reality, which

allows to display information of the digital twin next to the real twin. The machine is equipped with sensors

that continuously measure the state of the machine and are connected to the digital twin through EtherCAT

connection. The interactive system gives intuitive instructions that reduce the expert knowledge that is needed

such that even trainees can operate the machine and digitizes the process for experienced workers.

1 INTRODUCTION

The adjustment of a mechanical machine like a

tufting machine is a cumbersome task. Until today it

requires an iterative process of adjusting machine

parameters and then test the settings which prohibit

frequent changes of the machine e.g., to produce

different products. It also relies on the knowledge of

experts as the relationship between the adjustment of

levers, phases etc. and the change of the machine

tools is not easy to understand. However, the trend in

industry 4.0 are customizable products which require

frequent machine adjustments. Another challenge for

small and medium enterprises (SME) is an ageing

workforce due to changing demography and how to

conserve valuable expert knowledge.

The assistance system developed tries to optimize

this process by equipping a machine with sensors to

measure tool positions and mirroring tool positions to

the digital twin, a virtual model of the real machine,

and being able to save these settings to create a

database for different products and then compares the

current tool positions with the desired positions and

gives hints to the user what parameters need to

change. This way the expert’s knowledge can be

conserved. The hints are displayed exactly where the

user needs to change a parameter and shows how

much to change a certain parameter to achieve a

requested accuracy. The parameters can also be

optimized in the simulation and then transferred back

to the real machine. The assistance system yields

reproducible results.

The paper is organized as follows: First, a quick

overview of the related literature is given, then the

concept of the digital twin-based assistance system

for tufting machines is made. Next, the process of the

development of the digital twin is described. Then the

assistance system itself is presented. Difficulties and

benefits along with proposed future work are

discussed. The paper ends with a conclusion.

2 RELATED LITERATURES

(Grieves, 2014) first presented the Digital Twin (DT)

concept in 2003. In 2014, he published a white paper

Figure 1: The cyber-physical system "tufting machine" and

its components (G. Reinhart, 2015; Roßmann & Schluse,

2020).

372

Huesener, D. and Rossmann, J.

Assistance System for the Interactive Machine Adjustment (of a Tufting Machine).

DOI: 10.5220/0011320300003274

In Proceedings of the 12th Inter national Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH 2022), pages 372-378

ISBN: 978-989-758-578-4; ISSN: 2184-2841

on the Digital Twin, which he defined as a virtual

representation of a physical asset fed with data from

the real machine and sending data to the real twin.

The DT consists of the machine data (called Digital

Shadow) and a model of the machine, see Figure 1.

The virtual representation enhances

conceptualization, comparison, and collaboration in

production processes since it provides a more

intuitive perspective compared to 2D sketches or data

in tables or regular graphs. As sight is the most

important human sense, a realistic visual view is

valuable. Different users can easily classify the

current state of the machine which is also consistent

for different users.

A Cyber-Physical System (CPS) is the

combination of the physical asset and the Digital

Twin with its corresponding communication

infrastructure. CPS are important building blocks of

Industry 4.0. Digital twins of different machines can

communicate with each other. Application areas of

the Digital Twin include health monitoring,

production planning (PLM), and the design of new

products. An advantage over conventional planning

software is that all data related to one product is stored

together in one place (the Digital Twin). This

presentation has massively increased the value of the

data. Although simplified, it tries to model accurate

behavior of the real twin.

In (Werner Kritzinger et al., 2018) the terms

Digital Model, Digital Shadow, or Digital Twin are

used to classify the level of communication between

the digital and physical assets. Whereas a Digital

Model is independent of its real asset, a Digital

Shadow only receives data and only the Digital Twin

allows bidirectional communication between digital

and physical assets. However, “Digital Twin” is often

used for Digital Models and Shadows as well. The

literature on actual Digital Twins however is scarce.

Especially there is a lack of case studies on a higher

level of integration.

When combined with modern simulation

technologies (such as FEM, fluid dynamics,

multibody dynamics), the digital twin becomes an

experimentable Digital Twin (Schluse & Rossmann,

2016). The digital twin ensures that results of several

simulations are stored in the digital twin and thus

through co-simulation exchanged between them

thereby circumventing incorrect results if those

simulations were carried out independently. With the

introduction of simulation to the Digital Twin not

only visualization of the current data becomes

feasible but also the generation of new data, hence “a

look into the future”. It opens the digital twin to

artificial intelligence techniques that can optimize the

assets behavior.

As a mediator, the Digital Twin can process the

machine data, find suitable settings for a given task,

and display valuable information, that is otherwise

invisible, in an understandable way, such as

suggesting actions to be taken through a human-

machine interface. In (Cichon, 2019) a concept to

facilitate the interaction of humans and machines is

presented. Among others, joysticks, screens and

Augmented or Virtual Reality (AR/VR) allow direct

interaction with the Digital Twin. The user interacts

with the virtual machine like with the real machine

and can observe its status via the Digital Twin.

(Andre Schult et al., 2019) have developed an

assistance system that records machine data and then

uses machine learning algorithms to estimate the state

of the machine. This state is compared with a user

created database with common faults to detect faults

and give recommendations to fix those.

In the project virtual textile learning, (Haase et

al.) work on assistance systems in the textile sector.

Their focus is on using digitalization and 2D/3D

visualization for learning in the textile sector.

(Minoufekr et al., 2019) built an assistance system

based on the Microsoft HoloLens for CNC machine

tools that allows for much quicker and error tolerant

testing of the machining process. Like the tufting

machine used in our project, their model was based

on kinematic chains, that they modelled in the game

engine unity.

3 CONCEPTS

A tufting machine stitches yarn into a backing

material. It is commonly used to produce carpets or

artificial grass. A shaft continuously rotates and

thereby moves the tools. The tools commonly consist

of grippers, knives, and needles, although variations

exist. The needle stitches the yarn through a backing

material which is then grabbed by the gripper. When

the needle moves back up the knife cuts through the

loop formed by the gripper.

The assistance system proposed is based on an

experimentable digital twin of the tufting machine.

The experimentable digital twin was first presented in

(Hüsener et al., 2022). A digital model of the machine

was created that accurately describes the kinematics

of the machine. It is possible to adjust the machine

just like the real machine, but much simpler and much

more settings can be tried. The behavior of tools such

as needle, gripper and knife can be estimated.

Assistance System for the Interactive Machine Adjustment (of a Tufting Machine)

373

Different settings can be shown as ghosts (Figure

2) that allow to compare different settings in a way

not possible in the real machine.

Figure 2: "Digital ghosts": Simulation allows to visualize

effects of different settings.

Parameters are adjusted through sliders that when

hovered highlight which part will change when a

particular parameter is adjusted, see Figure 3. The

opacity of the machine case can be adjusted to show

what’s going on inside the machine.

Figure 3: Assistance system to adjust machine parameters

intuitively with immediate visibility of changes and the

option to return to a previous setting.

The machine was equipped with sensors that

continuously send their measurements to a PC where

the simulation runs (digital twin). The measurements

are then preprocessed, and tool positions are

calculated using kinematic relations between sensor

changes and tool position changes.

The simulation model combined with the real

machine data yields the experimentable digital twin

of the tufting machine.

To close the cycle from the machine to the

simulation back to the machine the assistance system

will guide the user to transfer settings stored in a

database or found in simulation back to the machine.

The benefit of the digital twin is that it constantly

receives new data so it can easily control the user has

made settings as desired. The accuracy can be

specified and is limited by the accuracy of the

sensors.

4 DEVELOPMENT OF THE

DIGITAL TWIN

The Digital Twin was modelled using VEROSIM, a

simulation software for rigid-body mechanics. The

software was developed with digital twins in mind

and has plugins for interfaces to popular

communication infrastructures, such as TCP/IP, ROS

or MQTT communication.

At first, the kinematic relations of a particular

tufting machine were measured at the real machine to

create a digital model of the tufting machine. The

simulation is already able to simulate the behavior of

the real machine with all the adjustments that can be

made on the real machine, but adjustments can be

made much quicker and easier and are reversible

allowing to test different machine settings.

In a second step, the machine was equipped with

sensors to measure the current positions of axis or the

phase of the main shaft. At first those values could be

stored inside an csv file that contains a timestamp and

each sensor value and then be read by the simulation

software which then shows a virtual representation of

that values. Multiple settings can be overlayed so

changes from one setting to another can be inherently

noticed.

Once the simulation can reproduce the measured

data, it can be used to (automatically or manually)

search for optimal settings for the intended use.

Third, an interface to directly receive values from

the sensors inside VEROSIM was written. The

interface uses ADS which is a communication

protocol by Beckhoff TwinCAT that supports real

time communication. The digital twin is notified

whenever a value changes and updates its

representation accordingly. If the machine is

available remotely, then the digital twin can be shown

on any computer.

To get the virtual representation of the current

state of the machine, a script was written that

transforms sensor values into tool positions. Not all

parameters can be measured with sensors, such that

manual measurements are necessary for machine

calibration. A dialog box allows the user to change

lever arms and offsets used for this calculation to

account for varying placement of the sensors.

Finally, if the user has adjusted the virtual digital

twin in simulation mode and wishes to transfer these

adjustments back to the real machine, this paper

proposes an assistance system that aims to achieve

just that.

Figure 4 shows all the system components of the

experimentable digital twin of the tufting machine.

SIMULTECH 2022 - 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications

374

Figure 4: Conceptual description of the system.

The user interacts through a human machine

interface with the digital twin. The digital twin

consists of a database that can store old machine

settings and a simulation engine that can find new

enhanced settings.

5 ASSISTANCE SYSTEM FOR

INTUITIVE ADJUSTMENTS

Handbooks specify how the curves of gripper on the

needle must look like. It must hit the needle at the

phase, see Figure 5. Similarly, handbooks give

specifications of the relative curves of gripper and

knife or the stroke of the needle. However, if changes

Figure 5: Drawing shows the desired curve representing

gripper and needle relative positions from handbooks.

are made to the machine, e.g., due to wear or

production changes and now different tools are

needed, then it not obvious which parameters need to

change how much to get as close as possible to the

desired setup.

So, normally it is necessary to make some

adjustments based on the experience of the operator,

test the new settings, and then readjust the machine if

necessary. This can be a costly task.

Simulation however can test various machine

settings with a fraction of the cost.

At first the needle is adjusted to yield just the right

amount of stroke, and then the gripper is adjusted

accordingly.

Figure 6: Sketch of needle with curves of relative gripper

positions for varying parameters.

Once the desired specifications for the needle

have been met, we can simulate various curves of the

gripper relative to the needle and compare the curves

with the desired result, see Figure 6. A MATLAB was

written that compares the curves considering the hit

point, the direction, and the distance from the loop to

recommend a curve. The simulation range and step

width can be adjusted to meet the requirements of the

operator. The settings for the knife are then found

similarly by considering relative movement between

gripper and knife. The process can be adapted to other

textile machines.

The process of finding optimal parameters will be

the subject of another paper. Instead, in this paper I

want to focus on how the experimentable digital twin

can help to adjust the real machine according to the

newly found optimized parameter sets.

If the operator has found optimal settings, he

starts the assistance system. The system can show all

the necessary steps on the digital twin, yet it is able to

also use the HoloLens to display the steps on the real

machine or let a trainee operate on a machine that

looks and works just like the real machine but is only

virtual, thereby enabling training without access to a

real machine.

Assistance System for the Interactive Machine Adjustment (of a Tufting Machine)

375

Figure 7: Process for the assistance system.

The process is shown in Figure 7. In the

initialization step a class containing among other

information the desired positions, a link to the

kinematics, and a camera view for each parameter is

created. “hideInstructions” makes sure that all panels

and arrows are hidden, e.g., if the adjustment process

is aborted.

Figure 8: Close-up view inside the machine showing pane

with instructions on how to adjust machine and an arrow

indicating that the operator should turn the excentre

clockwise.

The interactive part is started when the user

presses a predefined button. It then iterates over all

parameters and shows a board with the component to

change, the value the component needs to change and

an indication in which direction a value needs to

change, see Figure 8. It thereby transforms sensor

values and desired values from their mathematical

value to a meaningful representation for that machine,

e.g., move by 10° or move to notch 2/32. The needed

accuracy for continuous values was set to 0.1° or 0,5

mm but can be set according to the operator’s need.

The instructions are updated at every simulation step

to recalculate the required change as the operator

changes the value. If the desired and measured value

are within a tolerance the assistance system moves on

to the next unset parameter and repeats the process

until all parameters have their desired value.

6 DISCUSSION

Since many operators of tufting machines have

different backgrounds, it is expected that they can

benefit from an assistance system that gives intuitive

instructions how to adjust the tufting machine for a

given use case.

The experimentable digital twin has several

advantages over the real twin. Changes to machine

parameters are non-destructive, with one click an

adjusted value can be restored. The digital twin allows

simulation to search for optimal parameters, but it also

has access to the current state of the machine. Yet, it

can enhance the real machine by displaying additional

virtual information. Therefore, it combines features of

simulation and the real machine.

SOML Net structure of Instance Object Main

Begin

Init

Init_finished

hideInstructions

ready

keyPressed

Start

nextParam

adjust

finished updateInstructions

paramAdjusted

allSetnotAllSet

SIMULTECH 2022 - 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications

376

Benefits of using Twin CAT for the

communication between the sensors in the machine

and the digital twin are the frequent use of Twin CAT

in industry as well as the fast communication so that

the user gets feedback when he changes machine

values with little delay.

To ensure that the simulation produces viable

results, validation of the simulation model is necessary.

While some parts of the machine can be simplified it is

important that the tool movement is accurately

modeled so that the results found through simulation

can be transferred back to the machine, and if done so,

yield the expected results. To check visually that the

movement of the tools is similar, a camera was placed

in the machine and simulation, based on sensor data,

and the video can be compared, see Figure 9.

Figure 9: Camera and simulation alongside for validation.

For more accurate results the tool positions of the

simulation can be compared with the real tool

positions calculated from current sensor values. It

was found that at first the position was somewhat off,

but the simulation model was adjusted accordingly.

The real positions can be rendered on top, so at every

point in time it can be seen if the simulated and real

positions are close, see Figure 10. By calculating the

distance between two frames the error can be

quantified. It needs to be considered that the

movements of needle and gripper are idealized as

linear movements and only the linear component is

measured by the sensor.

Figure 10: Simulation of the tools with real tool positions

overlayed.

On the HoloLens the operator has his hands free

to adjust the machine but can see the instructions

simultaneously. Also, the HoloLens can be used to

visualize the tool movement with the current

parameters without turning the machine. In future, a

digital ghost of the target position of all machine parts

could also be shown alongside the instructions that

might even be more intuitively but when small

changes need to be made quantitative instructions

should yield more accurate results.

It is also suggested that the assistance system is

evaluated in practice, therefore it needs to be

extended for other set of tools or machines. Input

from industry experts can provide hints how the

usability of the assistance system might be enhanced.

It is expected that in the long run, an assistance

system can significantly reduce the time needed to

adjust a machine and thus reduce downtime and

ultimately cost. Evaluation could help quantify this

effect.

In further projects the experimentable digital twin

system shall be used for educational purposes to

demonstrate effects of errors in machine usage.

7 CONCLUSIONS

As was shown in this paper, manual tasks can benefit

from an assistance system that gives intuitive

instructions how to adjust machine settings for

different uses. An experimentable digital twin of a

tufting machine was developed that uses simulation

to find optimal adjustment parameters and a

simulation-based assistance system that uses real-

time sensor values to compare desired and actual state

and generate instructions to adjust the machine

accordingly. With time, the assistance system can be

extended with a database containing relevant settings

for different use cases. The benefits are repeatability,

reduced times to setup the machine, and a learning

environment for trainees. While the process was

shown and developed with a particular tufting

machine in mind, the system can be extended to other

tufting or textile machines with similar mechanics.

The main effort is in creating a digital twin – if a CAD

model of the machine already exists the system could

be adapted with minimal effort.

ACKNOWLEDGEMENTS

The writing of this chapter was enabled within the

context of the IGF project ‘Development of an

Assistance System for the Interactive Machine Adjustment (of a Tufting Machine)

377

experimentable digital twin for the analysis and

automated adaptation of textile manufacturing

processes using the example of tufting technology (T-

EXDIZ)’ 21166 N/2 of the Forschungsvereinigung

Forschungskuratorium Textil e.V.. It is funded via the

AiF within the framework of the program for the

promotion of joint industrial research and

development (IGF) by the Federal Ministry of

Economics and Energy based on a resolution of the

German Bundestag.

REFERENCES

Andre Schult, Lukas Oehm, Sebastian Carsch, Markus

Windisch, & Jens-Peter Majschak (2019). Selbst-

lernende Assistenzsysteme für Maschinenbediener.

Entwerfen Entwickeln Erleben EEE2019 Band 1, 159–

172.

Cichon, T. (2019). Der Digitale Zwilling als Mediator

zwischen Mensch und Maschine // The Digital Twin:

Mediator for Man-Machine Interaction [, RWTH

Aachen University]. Mendeley (via BibTeX-Export) //

DataCite.

G. Reinhart. (2015). Digital Twin - Synchronizing Reality

and Virtuality.

Grieves, M. (2014). digital twin: Manufacturing excellence

through virtual factory replication.

Haase, T., Termath, W., & Dick, M. Integration von VR-

und AR-Technologien in betrieblicher Lernphase.

Arbeitsalltag als Bildungsalltag. In Proceedings of

DELFI Workshops 2020. https://doi.org/10.18420/delfi

2020-ws-102

Hüsener, D., Schluse, M., Kaufmann, D., & Roßmann, J.

(2022). The Digital Twin as a Mediator for the

Digitalization and Conservation of Expert Knowledge.

In T. Schüppstuhl, K. Tracht, & A. Raatz (Eds.), Annals

of Scientific Society for Assembly, Handling and

Industrial Robotics 2021 (pp. 241–251). Springer

International Publishing. https://doi.org/10.1007/978-

3-030-74032-0_20

Minoufekr, M., Schug, P., Zenker, P., & Plapper, P. (2019).

Modelling of CNC Machine Tools for Augmented

Reality Assistance Applications using Microsoft

Hololens. In O. Gusikhin, K. Madani, & J. Zaytoon

(Eds.), Proceedings of the 16th International

Conference on Informatics in Control, Automation and

Robotics // ICINCO 2019: Proceedings of the 16th

International Conference on Informatics in Control,

Automation and Robotics: Prag, Czech Republic, July

29-31, 2019 (pp. 627–636). SCITEPRESS - Science

and Technology Publications; SCITEPRESS - Science

and Technology Publications Lda. https://doi.org/10.52

20/0007920806270636

Roßmann, J., & Schluse, M. (2020). Experimentierbare

Digitale Zwillinge im Lebenszyklus technischer

Systeme. In W. Frenz (Ed.), Handbuch Industrie 4.0:

Recht, Technik, Gesellschaft (pp. 837–859). Springer

Berlin Heidelberg. https://doi.org/10.1007/978-3-662-

58474-3_43

Schluse, M., & Rossmann, J. From simulation to

experimentable digital twins: Simulation-based

development and operation of complex technical

systems. In 2016 IEEE International Symposium on

Systems Engineering (ISSE) (pp. 1–6).

https://doi.org/10.1109/SysEng.2016.7753162

(Original work published 2016)

Werner Kritzinger, Matthias Karner, Georg Traar, Jan

Henjes, & Wilfried Sihn. (2018). Digital Twin in

manufacturing: A categorical literature review and

classification. In Elsevier (Ed.), 16th IFAC Symposium

on Information Control Problems in Manufacturing

INCOM 2018 (51st ed., Vol. 51, pp. 1016–1022). IFAC

Secretariat. https://doi.org/10.1016/j.ifacol.2018.08.474

SIMULTECH 2022 - 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications

378