Multimodal Feedback Estimation for Knob Interactions in Virtual
Reality for Control Panels
Valentina Gorobets
1 a
, Long Cheng
2 b
, Helene Lussi
1
and Andreas Kunz
1 c
1
Institute for Machine Tools and Manufacturing, Swiss Federal Institute of Technology Zurich,
Clausiusstrasse 33, Zurich, Switzerland
2
Digital Innovation Lab, RhySearch, Buchs, Switzerland
Keywords:
Virtual Reality, Haptic Feedback, Knob Interaction, Control Panels.
Abstract:
This paper presents findings on the unimodal and multimodal feedback design for the interaction with a virtual
knob. Since the physical knob provides haptic feedback while being rotated, we also integrated haptic feedback
in the virtual knob. The real and virtual knob consist of a main body and a handle on top. For fine and coarse
adjustments, the knob can be grabbed and rotated with a ’Grasp’ or a ’Pinch’ gesture. In a user study with 30
participants, we evaluated our system using objective measures and subjective metrics. The results show that
participants reported a preference for having a haptic feedback and perceived it as more natural.
1 INTRODUCTION
Virtual Reality (VR) is used in many fields, such as
industrial training (Hirt et al., 2020). Here, physi-
cal control panels are required, where knobs and but-
tons are used to adjust parameters. While some real
knobs have mechanisms that cause them to “snap” at
specific increments, implementing appropriate hap-
tic feedback in VR is still challenging. Given the
currently existing haptic feedback capabilities of VR
controllers, researching button and knob interactions
with combinations of different feedback and snapping
mechanisms gives valuable insights.
2 RELATED WORK
Approximately 80% of information is acquired
through sight, about 10% through hearing, and the re-
maining 10% through the other channels (Man and
Olchawa, 2018). Consequently, there is an interest in
providing these sensory inputs in VR to create an in-
creased sense of presence (Gallace et al., 2012). How-
ever, feedback systems in consumer-segmented VR
devices are primarily limited to visual and audio cues.
More recent VR controllers are equipped with hap-
a
https://orcid.org/0000-0002-8615-5972
b
https://orcid.org/0009-0003-0005-4148
c
https://orcid.org/0000-0002-6495-4327
tic feedback mechanisms and can additionally pro-
vide vibrotactile feedback to users through their hands
(Kreimeier et al., 2019).
(Tatzgern and Birgmann, 2021) used virtual hand
and raycasting techniques to explore input modalities
on VR control panels. They identified knobs, buttons,
and sliders as control elements and compared three
interaction methods for manipulation using a VR con-
troller. The interaction possibilities consist of a trig-
ger, a joystick, and an approximating hand gesture
method. The joystick performed best for slider ma-
nipulations due to its high precision, while the ap-
proximate hand gesture was preferred for knobs but
did not outperform the trigger method.
Physical proxies can be used to provide haptic
feedback when using hand-tracking. This technique
was also employed for control panel interactions.
(Matthews et al., 2023) designed a physical control
panel consisting of buttons, sliders, and knobs and im-
plemented a retargeting technique to map only a third
of the physical panel to the entire virtual interface.
They observed a decrease in performance when users
interacted with the remapped interface.
Instead of attempting to provide all possible types
of feedback, researchers propose an approach that
integrates task-related information through the best
matching one. (Cooper et al., 2018) designed a vir-
tual scenario where participants performed a wheel
change in VR while wearing vibration gloves and
holding a mock-up wrench. Multisensory cues in
588
Gorobets, V., Cheng, L., Lussi, H. and Kunz, A.
Multimodal Feedback Estimation for Knob Interactions in Virtual Reality for Control Panels.
DOI: 10.5220/0013180200003912
Paper published under CC license (CC BY-NC-ND 4.0)
In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2025) - Volume 1: GRAPP, HUCAPP
and IVAPP, pages 588-595
ISBN: 978-989-758-728-3; ISSN: 2184-4321
Proceedings Copyright © 2025 by SCITEPRESS – Science and Technology Publications, Lda.

form of visual, audio and tactile feedback were used
to compensate for sensations that are not easily repli-
cable in VR. Results show that the multimodal sub-
stitute feedback improves overall task performance.
These conclusions align with previous findings sug-
gesting that training with low fidelity simulators is not
inferior to high fidelity ones (Lefor et al., 2020).
Research shows that using multimodal feedback
may not uniformly improve performance and can in-
crease perceived task load and fatigue. For instance,
(Faeth and Harding, 2014) found that, while the bi-
modal feedback system outperformed the unimodal
one, the application of trimodal feedback in virtual
button interactions resulted in decreased performance
(Bermejo et al., 2021). This occurrence of overstim-
ulation has been reported in other studies as well
(Cooper et al., 2018), (Apostolou and Liarokapis,
2022), (Marucci et al., 2021), indicating that unneces-
sary or incongruent feedback can disrupt immersion.
Marucci et al. (Marucci et al., 2021) explored the
impact of multisensory feedback and perceptual load
on performance, workload, and presence. Participants
engaged in the same task under two conditions: one
with high perceptual load and another with low per-
ceptual load, and with either only visual or with ad-
ditional audio and/or vibrotactile feedback. Results
show, that only in the high load condition, multisen-
sory (bi- or trimodal) stimuli significantly enhanced
performance compared to visual stimulation alone.
3 METHODOLOGY
3.1 Hardware and Software
The feedback system for knob interactions was de-
veloped and assessed using the Meta Quest 2 oaired
with the corresponding controllers. They provide vi-
brotactile feedback and are equipped with capacitive
sensors on the buttons, joystick, touchpad, index trig-
ger, and middle finger trigger, facilitating partial fin-
ger and thumb tracking.
The implementation was developed in Unity
2022.3.9.f1 as a standalone application. We used
the Oculus Interaction SDK for hand prefabs, con-
troller inputs, the VR camera rig, and anchor points.
The Quick Outline Asset
1
was used to integrate wire-
frames as visual feedback. Blender and Shapr3D are
used to create the virtual mock-up (Fig. 1(b)) of the
physical knob (Fig. 1(a)) on the control panel.
1
https://assetstore.unity.com/packages/tools/particles-
effects/quick-outline-115488, accessed 16.01.24
Figure 1: (a) Physical knob. (b) Virtual knob.
3.2 Knob Interaction and Feedback
The design of the virtual knob and the implementa-
tion of its interactions replicates the knob from a real
control panel. The knob is used to control the position
of the machine axis and allows two distinct manipu-
lation possibilities: the body can be gripped using a
precision grip and rotated by twisting the fingers. Al-
ternatively, it can be pinched at the handle and rotated
through circular motions of hand and forearm. The
different hand gestures have designated purposes; a
precision grip allows for slow and precise rotation,
whereas the pinching gesture and applied circular mo-
tion allow for faster rotation. The chosen hand ges-
ture also influences the rotation resistance of the knob.
When force is applied to the knob at the handle, it
initially presents a notable resistance, demanding a
force to overcome the threshold for rotation. How-
ever, once surpassed, the knob smoothly rotates with-
out providing the haptic feedback of detents. In con-
trast, when the knob is rotated at the body, it exhibits
a low torque, enabling slow and precise manipulation
with rotational force feedback of individual detents.
These two interaction possibilities for the virtual
knob are implemented following a concept similar
to the approximation of hand gestures proposed by
(Tatzgern and Birgmann, 2021). Two specific button
combinations are used for those two interaction possi-
bilities. The precision grip is approximated by press-
ing both the trigger and grasp buttons on the controller
and keeping the thumb on the joystick. The pinching
gesture is approximated by pulling the trigger button.
We will refer to them as ’Grasp’ and ’Pinch’ further
in the text. Additionally, our virtual knob implemen-
tation consists of two so-called activation zones for
each of the interaction possibilities. The start of each
interaction with the body or the handle of the knob
can be triggered only in the corresponding activation
zone. They are realized as invisible cylinders. For
pinching the handle, the middle point between the in-
dex finger and thumb needs to be within the handle ac-
tivation zone (diameter: 25mm, height: 70mm), and
the trigger button must be pressed. To grab the knob’s
body, the center point of the virtual palm must be
within the cylinder (diameter: 70mm, height: 70mm),
positioned above the knob’s body and handle. Simul-
taneously pressing the trigger and grasp button in this
Multimodal Feedback Estimation for Knob Interactions in Virtual Reality for Control Panels
589

zone will result in the knob body being grasped. This
is done to minimize the snapping effects and to en-
courage users to aim at a specific part of the knob.
We designed three vibrotactile feedback modali-
ties for the interaction with the virtual knob. Table 1
shows all knob conditions studied. We refer to ’NF’
as no additional visual and no vibrotactile feedback
for the virtual knob.
Table 1: Description of all knob conditions.
Configuration Visual Vibrotactile
NF No No
SVT No Simple
CVT No Complex
V Yes No
V+SVT Yes Simple
V+CVT Yes Complex
To simulate the detents of the physical knob, a
snapping effect is added to the virtual knob. We set
the angular resolution of the knob to 10°, staying
within the recommended 5°range by (Hinricher et al.,
2023). During our implementation, we noticed that
setting the vibration amplitude to 0.3 of the maximum
amplitude and the duration of each pulse to 0.01s
gave a natural feel both for the slow knob rotation as
well as for a faster one. While a higher amplitude
provided a stronger clicking sensation which benefits
the slow knob turning, it resulted in an uncomfort-
able strong vibration during fast turns. Therefore, we
implemented two types of vibrotactile feedback: one
with the maximum amplitude (referred to as ’simple’
(SVT)) and one with varying amplitude (referred to
as ’complex’ (CVT)).
Additionally, for half of the conditions, we imple-
mented visual feedback (referred to as V) in the form
of a highlighting wireframe around the knob body and
handle. Whenever the hand or fingers were within the
grab zone and in a position to grasp either the body or
the handle, or both parts, the respective components
were highlighted.
3.3 User Study
For the user study, participants were standing in front
of the control panel while operating the virtual knob.
The design of the virtual control panel is derived from
the control panel of a machine tool that we had avail-
able on-site. Similarly to the real control panel, we
designed the virtual control panel with an inclination
angle of 60°.
We designed a simplified scenario in which the
knob is rotated to change the position of a cube. The
task was to move the cube to a predefined target po-
sition. The participants were assigned the same task
across all conditions (Tab. 1). The small cube was
positioned on top of the control panel, and the target
location was indicated by a red square. Rotating the
knob to the right moved the cube rightward and con-
versely for leftward rotation. Upon successful place-
ment, the target color changed to green, as shown in
Figure 2.
Figure 2: The cube is moved to a red square target, turning
green upon success. (a) Start position, (b) Task completed.
As introduced in Table 1, the user study consists
of six feedback varieties: no feedback (NF), visual
feedback (V), simple vibrotactile feedback (SVT),
complex vibrotactile feedback (CVT), visual and sim-
ple vibrotactile feedback (V+SVT), and visual and
complex vibrotactile feedback (V+CVT). In the user
study, the participants can use both manipulation ges-
tures to interact with all six types of knobs. For each
feedback variant, three tasks are presented:
• The target position is 1.5cm away from the cube.
Participants use the pinching gesture to move the
cube to target.
• The target position is 1.5cm away from the cube.
Participants only use the grasping gesture to move
the cube to target.
• The target position is 20cm away from the cube.
Both gestures can be used.
The study started with welcoming participants and
filling out a consent form. After introducing the goal
of the study, they could test the physical knob. Af-
terwards, they filled out the pre-questionnaire includ-
ing demographics and personal well-being before the
study. The VR session started with a tutorial on the
interaction with the virtual knob. No data is collected
during the tutorial. After completing the tutorial, par-
ticipants performed the tasks in a randomized order.
They were instructed to complete the tasks as fast as
possible and with as few errors as possible. Upon
completion, a final scene was loaded where the user
could experience all types of feedback implemented
for the knobs without any time constraints for a better
subjective comparison.
We measured task completion time (TCT), error
count (EC), and overshoot distance (OD) as objective
measurements. The TCT measures the time until a
correct placement was reached. The timer starts when
the knob is first grabbed and stops once the knob is
released. The EC is the number of times the cube
passes the target location in both forward and back-
HUCAPP 2025 - 9th International Conference on Human Computer Interaction Theory and Applications
590

wards direction. For every error, the distance of the
target over- or undershot is measured. For subjective
measurements, participants’ well-being was assessed
through the Simulator Sickness Questionnaire (SSQ)
(Kennedy et al., 1993). The Presence questionnaire
(Slater et al., 1999), NASA Task Load Index (NASA-
TLX) (Hart and Staveland, 1988), and System Usabil-
ity Scale (SUS) (Brooke, 1996) are also included.
We formulate the following hypotheses regarding
the impact of multimodal feedback on user perfor-
mance and preferences:
• H1: Vibrotactile feedback or visual feedback,
alone or combined, does not improve perfor-
mance.
• H2: User has no preference over different types
of feedback.
• H3: Using grasping or pinching gesture has no
effect on performance.
We used R for the statistical analysis. The signifi-
cance level is set 0.05. In results section, We analyzed
and ran statistical tests on task performance (TCT, EC
and OD) and subjective responses for different ges-
tures and different feedback modalities.
4 RESULTS
30 participants (19 male, 11 female) aged between 20
and 31 were recruited. 16 participants had less than
20 hours of VR experience, and 7 were using VR for
the first time. Three subjects had 20 - 100 hours, and
four had more than 100 hours of VR experience.
4.1 Objective Performance
Feedback Modalities. For the pinching gesture, SVT
and CVT provide knob vibration. The vibration inten-
sity in CVT only changes to max for the grasp ges-
ture. Figure 3 (a) - (c) shows the user performance
(TCT, EC, OD) of the different conditions for pinch-
ing gesture. The data did not meet the assumption
of normality, as indicated by p-values smaller than
0.05 obtained from the Shapiro-Wilk test. The non-
parametric Friedman tests show no significant results:
p
TCT
= .1886, p
EC
= .5663, p
OD
= .9694. Figure 3
(d) - (f) shows the user performance under the dif-
ferent feedback conditions for grasping gesture. The
Friedman tests show no significant results: p
TCT
=
.9257, p
EC
= .4853, p
OD
= .8415. When users are
free to choose gesture(s), the Friedman tests show
no significant results in task completion, error rate,
and overshoot distance: p
TCT
= .7941, p
EC
= .3822,
p
OD
= .7504 (Figure 3 (g) - (i)). This indicates that no
significance for the feedback (visual or vibrotactile) is
found for all knob conditions.
Interaction Gestures. Each user tried grasping,
pinching or both gesture to complete the task. The
non-parametric Friedman tests showed no statistical
significance (Table 1). When participants were free to
choose between grasping and pinching gestures, they
used the ’Pinch’ gesture 85.2% and the ’Grasp’ ges-
ture 14.8% of the time. In 47.8% of the tasks, the
’Grasp’ gesture was never used. 3 participants (10%)
completed their first task (cube distance = 15mm
(Figure 2 (a)) using only the grasp gesture. After-
wards, like the other 27 participants, they began with
the pinching gesture and in some cases, eventually
switched to a grasp once the cube got close to the
target or they overshot the target with the pinching
gesture.
Table 2: p-Values from the Friedman tests when comparing
the performance between ’Pinch’ and ’Grasp’.
p
TCT
p
EC
p
OD
NF .273 .841 1
SVT 1 .297 .297
CVT .715 .819 .297
V .465 .683 .683
V+SVT .715 .127 .251
V+CVT .068 .67 1
4.2 Subjective User Experience
Knob Evaluation. Participants rated the feedback of
the knobs on a 7-point scale (4 = neutral ). To deter-
mine if the obtained scores significantly differ from
the neutral score, we conduct two-tailed one-sample
Wilcoxon signed-rank tests with the same confidence
level of 0.05. The hypothesis (H2) states that the
median of the population from which the sample is
drawn, equals the neutral score (µ
0
= 4).
Visual Highlighting. The results of highlighting in-
dicates that participants found the highlighting of the
knob parts to be helpful, with a significant higher rat-
ing than 4 (p = .037) and an average score of 4.73.
While the highlighting was not perceived as distract-
ing, as reflected by the significantly higher rating than
4 (p = .001) and a mean score of 5.5, participants had
varying perceptions of its realism, yielding in an av-
erage score of 4.03 and p = .908.
Vibration. Participants perceived the knob vibrations
as highly helpful, with a mean score of 5.77 and a
significantly higher rating (p = 3.33 − 05). Gener-
ally, participants did not perceive the vibrations to be
distracting, as indicated by a mean score of 5.13 and
p = .015. However, the high SD of 2.26 suggests vari-
ability in responses, with 4 subjects finding the vibra-
Multimodal Feedback Estimation for Knob Interactions in Virtual Reality for Control Panels
591

(a) (b) (c)
(d) (e) (f)
(g) (h) (i)
Figure 3: Task performance. (a) TCT with pinching gesture. (b) EC with pinching gesture. (c) OD with pinching gesture. (d)
TCT with grasping gesture. (e) EC with grasping gesture. (f) OD with pinching gesture. (g) TCT with both gestures. (h) EC
with both gestures. (i) OD with both gestures.
tion to be extremely distracting and another 4 find-
ing it moderately distracting. Moreover, the knobs
equipped with vibration were perceived as realistic
and significantly rated higher than 4 (p = 4.37e−05),
with an average score of 5.47. As for the question
comparing vibration and non-vibrating knobs, most
participants perceived the latter as less realistic com-
pared to the vibrating ones, as indicated by the aver-
age score of 2.67 and p = .002.
Standardized Questionnaires. The average SUS
score is 84.5 with an SD of 11.93. The final score
of the Presence Questionnaire is on average 4.52 with
an SD of 1.184. The average overall workload score is
17.37 with an SD of 9.32.(Hart and Staveland, 1988).
The results of the SSQ pre- and post-study indicate
that 10 participants had higher scores after the study.
Among them, eight participants had score differences
below 10, one had a difference of 11.22, and an-
HUCAPP 2025 - 9th International Conference on Human Computer Interaction Theory and Applications
592

other one had a difference of 22.44. On average, the
score of the pre-questionnaire was 15.334 with SD =
21.593, and for the post-questionnaire, the mean was
14.212 with SD = 20.09. The average difference is
-1.122 with SD = 10.766. These results indicate that
there were no remarkable differences in the partici-
pants’ well-being before and after the study.
5 DISCUSSION
The results from our VR knob implementation ques-
tionnaire reveal a positive user experience. Partici-
pants reported a moderately high presence score, in-
dicating a satisfactory level of immersion. Addition-
ally, the implementation generated minimal perceived
workload, minimal simulator sickness, and is user-
friendly, providing an engaging and comfortable in-
teraction with the VR knobs.
5.1 User Evaluation of Knob Feedback
Although the performance was not significantly im-
pacted by the knob feedback, our analysis revealed
significant findings from the knob questionnaire that
participants generally favored knob feedback. A
summary of the results obtained by the one-sample
Wilcoxon signed-rank tests can be seen in Table 3.
5.1.1 Visual Feedback
Although wireframe highlighting is not present in the
real world, participants rated its realism relatively
neutral. This suggests that visual feedback by a wire-
frame might disturb surface fidelity, but won’t signifi-
cantly increase the perceived workload. Whether such
visual feedback could lower the workload depends on
personal preference, as half of the participants found
highlighting very helpful, while the rest ranked it as
not helpful or more or less neutral. This is also re-
flected in the knob rating inside the virtual environ-
ment, where 43.3% found the knob with visual feed-
back easier to control, 26.7% preferred the knob with
no feedback, and 30% had no preference.
5.1.2 Vibrotactile Feedback
The knob vibration was rated as very helpful. This is
supported by the rating in the VR scene, with 86.7%
rating the knob with SVT feedback as easier to con-
trol. Our findings suggest that vibrotactile feedback
technique can assist users in completing their tasks,
but it could also lead to a higher perceived workload
depending on the individual. Our results indicate that
vibration pulses are an appropriate way to mimic knob
detents. This suggests that when using the grasping
gesture, a higher vibration intensity can be perceived
as more beneficial, without impacting user experience
in terms of distraction level and realism. However, the
answers from the questionnaire contradict to the par-
ticipants’ rating inside the VR scene, as 46.7% found
the knob with strong vibration easier to control and
40% the knob with weak vibration.
5.2 User Performance
We noticed throughout the study, that some partic-
ipants encountered the same issue: once the cube
reached its correct position, they started moving their
hand away from the knob before the virtual hand fully
released the grip, causing the cube to unintention-
ally move by one unit distance. This led to a cy-
cle where attempts to correct overshoots resulted in
undershoots and vice versa, significantly increasing
both, task completion time and error count. We ob-
served this particularly among inexperienced users.
Thus, in hindsight, our implementation may have
been overly sensitive to user input, despite efforts to
address this by introducing an angular resolution of
10°. The problem could have been avoided, for exam-
ple, by providing a slightly larger target for the cube,
where one over- or undershoot would still be consid-
ered correctly placed.
1. “Visual feedback does not improve user per-
formance compared to no feedback”. Our re-
sults across all three manipulation conditions (Pinch,
Grasp, Both) and performance metrics (TCT, EC,
OD) indicate that participants performed similarly in
both the NF and V conditions. This outcome could be
explained by participants’ familiarity with the grab-
bing process, as it was introduced during the tutorial,
suggesting that the additional visual feedback might
not have significantly altered their performance.
2. “Vibrotactile feedback does not improve user
performance compared to no feedback”. The
results indicate that participants performed equally
when any type of vibration (SVT or CVT) was present
compared to no feedback (NF). While not significant,
we can observe that with the ’Pinch’ gesture the per-
formance tends to be slightly worse when vibration
is present, as participants overshot more often and by
a lot more under the conditions SVT and CVT com-
pared to NF. These results could be attributed to the
task’s simplicity, aligning with Marucci et al.’s re-
search (Marucci et al., 2021), which suggests that ad-
ditional feedback modalities primarily enhance per-
formance when the task requires a high perceptual
load.
Multimodal Feedback Estimation for Knob Interactions in Virtual Reality for Control Panels
593

Table 3: Summary of the results for user evaluation of knob feedback that consists of visual and vibrotactile feedback.
Question Hypothesis Interpretation Type of feedback
1a Reject H
0
The Helpfulness score of the highlight mechanism is signifi-
cantly higher than the neutral score 4.
Visual
1b Reject H
0
The Distraction score of the highlight mechanism is signifi-
cantly higher than the neutral score 4.
Visual
1c Accept H
0
The Realism score of the highlight mechanism is not signifi-
cantly different from the neutral score 4.
Visual
2a Reject H
0
Helpfulness score of the vibration mechanism is significantly
higher than the neutral score 4.
Vibrotactile
2b Reject H
0
Distraction score of the vibration mechanism is significantly
higher than the neutral score 4.
Vibrotactile
2c Reject H
0
Realism score of the vibration mechanism is significantly
higher than the neutral score 4.
Vibrotactile
3 Reject H
0
Realism of vibration compared to no vibration is significantly
higher than the neutral score 4.
Vibrotactile
5a Reject H
0
Helpfulness of strong vibration compared to weak vibration
is significantly higher than the neutral score 4.
Vibrotactile
5b Accept H
0
Distraction of strong vibration compared to weak vibration
is not significantly different from the neutral score 4.
Vibrotactile
5c Accept H
0
Realism of strong vibration compared to weak vibration is
not significantly different from the neutral score 4.
Vibrotactile
3. “Visual and vibrotactile feedback combined
does not improve user performance compared to
no or only one type of feedback”. We determined
that neither visual (V) nor vibrotactile (SVT, CVT)
feedback improved user performance. Upon compar-
ing the unimodal conditions to the bimodal condi-
tions (V+SVT, V+CVT) for the ’Pinch’ gesture, we
noted that with bimodal feedback participants com-
pleted tasks slightly faster than in the SVT and CVT
conditions only. Additionally, they exhibited lower
overshoot distance. Similarly, using the ’Grasp’ ges-
ture with bimodal feedback resulted in slightly faster
task completion compared to only vibrotactile feed-
back (SVT, CVT).
4. “Strong vibration does not improve user perfor-
mance compared to weak vibration”. In our exper-
iment, weak and strong vibrations were exclusively
compared with the ’Grasp’ gesture, as we noticed an
uncomfortable strong vibration during fast turns with
the pinching gesture. Our data shows that there is no
noticeable performance difference between SVT and
CVT or V+SVT and V+CVT, suggesting that weak
vibrations might suffice to simulate the detents of the
knob.
5. “’Grasp Rotation’ does not allow for finer motor
control compared to ’Pinch Rotation’, thus, won’t
lead to a lower EC and OD”. Although there is no
statistical evidence, our results align with this hypoth-
esis. While subjects did not exhibit fewer errors with
the ’Grasp’ gesture, we observed a higher overshoot
distance for the ’Pinch’ gesture. Additionally, there
were instances of notably high overshoots with the
pinching gesture, indicated by the high standard de-
viations ranging between 3.99 and 15.27.
6 CONCLUSION
In this paper, we introduced a knob implementation
with visual and haptic feedback and proposed two dif-
ferent interaction possibilities, i.e., grasping the body
of the knob and pinching the handle. The pinching
and grasping gestures are triggered through different
controller inputs that approximate the same gesture
in real life. We explored the effects of no feedback
compared to unimodal feedback and bimodal feed-
back. Visual feedback is implemented as a wireframe,
highlighting the graspable knob part that appears once
the virtual hand approaches the knob to assist the
user in aiming. Vibrotactile feedback is realized to
mimic the haptic feedback that a real knob provides
while being manipulated. Different vibration inten-
sities were investigated by implementing interactions
with the knob with low and high vibration amplitude.
We conducted a user study that revealed a positive
user experience. Overall, the participants deemed the
proposed feedback cues as appropriate and showed a
preference for knobs with feedback. However, objec-
tive measurements did not yield significant results, in-
dicating that visual, vibrotactile, or their combination
had no effect on user performance.
HUCAPP 2025 - 9th International Conference on Human Computer Interaction Theory and Applications
594

6.1 Limitations and Future Work
Our implementation of the knob manipulation is sen-
sitive to hand motions. That caused some overshoots
and additional errors, when participants were placing
the cube to the target position. This can be improved
in two possible ways: either by decreasing the sensi-
tivity of the knob by increasing the angular resolution,
or by creating a slightly bigger target area to avoid
those overshoots that are caused by a small hand mo-
tion before the release of the virtual knob.
REFERENCES
Apostolou, K. and Liarokapis, F. (2022). A systematic re-
view: The role of multisensory feedback in virtual
reality. In 2022 IEEE 2nd International Conference
on Intelligent Reality (ICIR), pages 39–42, New York,
NY, USA. IEEE.
Bermejo, C., Lee, L. H., Chojecki, P., Przewozny, D., and
Hui, P. (2021). Exploring button designs for mid-
air interaction in virtual reality: A hexa-metric eval-
uation of key representations and multi-modal cues.
Proceedings of the ACM on Human-Computer Inter-
action, 5(EICS):1–26.
Brooke, J. (1996). Sus: a “quick and dirty’usability. Us-
ability evaluation in industry, 189(3):189–194.
Cooper, N., Milella, F., Pinto, C., Cant, I., White, M., and
Meyer, G. (2018). The effects of substitute multisen-
sory feedback on task performance and the sense of
presence in a virtual reality environment. PloS one,
13(2):e0191846.
Faeth, A. and Harding, C. (2014). Emergent effects in mul-
timodal feedback from virtual buttons. ACM Trans-
actions on Computer-Human Interaction (TOCHI),
21(1):1–23.
Gallace, A., Ngo, M. K., Sulaitis, J., and Spence, C. (2012).
Multisensory presence in virtual reality: possibilities
& limitations. In Multiple sensorial media advances
and applications: New developments in MulSeMedia,
pages 1–38. IGI Global, Hershey, PA, USA.
Hart, S. G. and Staveland, L. E. (1988). Development of
nasa-tlx (task load index): Results of empirical and
theoretical research. In Advances in psychology, vol-
ume 52, pages 139–183. Elsevier, Paris, France.
Hinricher, N., Schr
¨
oer, C., and Backhaus, C. (2023). Design
of control elements in virtual reality—investigation
of factors influencing operating efficiency, user ex-
perience, presence, and workload. Applied Sciences,
13(15):8668.
Hirt, C., Spahni, M., Kompis, Y., Jetter, D., and Kunz, A.
(2020). Virtual reality training platform for a com-
puter numerically controlled grinding machine tool.
International Journal of Mechatronics and Manufac-
turing Systems, 14(1):1–17.
Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilien-
thal, M. G. (1993). Simulator sickness questionnaire:
An enhanced method for quantifying simulator sick-
ness. The international journal of aviation psychol-
ogy, 3(3):203–220.
Kreimeier, J., Hammer, S., Friedmann, D., Karg, P., B
¨
uhner,
C., Bankel, L., and G
¨
otzelmann, T. (2019). Evaluation
of different types of haptic feedback influencing the
task-based presence and performance in virtual real-
ity. In Proceedings of the 12th acm international con-
ference on pervasive technologies related to assistive
environments, pages 289–298, New York, NY, USA.
ACM.
Lefor, A. K., Harada, K., Kawahira, H., and Mitsuishi, M.
(2020). The effect of simulator fidelity on procedure
skill training: a literature review. International jour-
nal of medical education, 11:97.
Man, D. and Olchawa, R. (2018). The possibilities of using
bci technology in biomedical engineering. In Biomed-
ical Engineering and Neuroscience: Proceedings of
the 3rd International Scientific Conference on Brain-
Computer Interfaces, BCI 2018, March 13-14, Opole,
Poland, pages 30–37, Cham, Switzerland. Springer.
Marucci, M., Di Flumeri, G., Borghini, G., Sciaraffa, N.,
Scandola, M., Pavone, E. F., Babiloni, F., Betti, V., and
Aric
`
o, P. (2021). The impact of multisensory integra-
tion and perceptual load in virtual reality settings on
performance, workload and presence. Scientific Re-
ports, 11(1):4831.
Matthews, B. J., Thomas, B. H., Von Itzstein, G. S., and
Smith, R. T. (2023). Towards applied remapped
physical-virtual interfaces: Synchronization methods
for resolving control state conflicts. In Proceedings of
the 2023 CHI Conference on Human Factors in Com-
puting Systems, pages 1–18, New York, NY, USA.
ACM.
Slater, M. et al. (1999). Measuring presence: A re-
sponse to the witmer and singer presence question-
naire. Presence: teleoperators and virtual environ-
ments, 8(5):560–565.
Tatzgern, M. and Birgmann, C. (2021). Exploring input
approximations for control panels in virtual reality.
In 2021 IEEE Virtual Reality and 3D User Interfaces
(VR), pages 1–9, New York, NY, USA. IEEE.
Multimodal Feedback Estimation for Knob Interactions in Virtual Reality for Control Panels
595