Comparative Study of ORB-SLAM2 and IR-Based Revolution

Counting with Ultrasonic Obstacle Avoidance for Autonomous

Wheelchair Navigation

Shreyas Anchan

, Giriprasath P

, Lohith M

, Rahul M

, Sharath Kumar Y N

and Satish B A

Department of Electrical and Electronics Engineering, Dayananda Sagar College of Engineering, Kumaraswamy Layout,

Bangalore, Karnataka, India

Keywords: Autonomous, Embedded systems, Navigation, Robotics, Wheelchair.

Abstract: Wheelchairs are crucial to the mobility of people with disabilities. Conventional manual wheelchairs have

come a long way to the powered and smart designs that allow for different types of controls. Nevertheless,

indoor navigation and evasion of obstacles is a challenge to most users, especially to those with disabilities

on powered wheelchairs. With technological advancement, the use of autonomous systems has gotten wider

scope incorporating both external and internal spaces. Moreover, nowadays use of expensive lidars is used

for indoor navigation but that makes the entire wheelchair costly. To reduce the cost use of stereo cameras or

monocular cameras started for indoor mapping. ORB-SLAM2 is an algorithm where VSLAM works using a

monocular camera because of which the cost of the wheelchair manufacture reduces.

1 INTRODUCTION

Wheelchairs have been used by the physically

challenged for movement for many years. The simple

manually operated wheelchairs have in recent years

given way to powered wheelchairs, with various

modes of control. Nowadays, wheelchairs have

features like stair climbing and autonomous

navigation built in the wheelchairs. But the cost of

these wheelchairs is high and not only that these

wheelchairs cannot be transported easily

SLAM has been a field of study for many decades.

From radars and range finders to cameras and lasers,

many modalities of SLAM-finding sensors pose a

global representation. Development of Lidar sensors

made major developments in SLAM-based

localization since these Lidars can give an accurate

measure of the z-axis distance of its surroundings.

https://orcid.org/0009-0005-6195-7257

https://orcid.org/0009-0005-3119-3524

https://orcid.org/0009-0001-8391-7935

https://orcid.org/0009-0001-6592-9166

https://orcid.org/0000-0002-1066-5773

https://orcid.org/0000-0003-4010-0965

There have been many developments in the

algorithms used in SLAM like ORB-SLAM2 and

ORB-SLAM3, Vins-Fusion, DSO (Direct Sparse

Odometry), RTAB-Map and LSD-SLAM, and

Kimera. Nowadays VSLAM (Visual-Inertial SLAM)

has become a major topic of study due to its

applications in robotics, avionics, AR/VR, etc.

In most cases, custom-made wheelchair motors

are used which are quite costly. In many scenarios,

motors that are used in the electric bicycles are

employed but they have a much higher rated rpm than

what is efficient and smooth operation of the

wheelchair requires. As Parik (Parikh et al., 2005)

implemented in the Usability study of a control

framework for an intelligent wheelchair, robotics to a

wheelchair. The study's objective is to combine three

approaches to motion control to evaluate the

usefulness and efficacy of three paradigms, namely,

deliberative plans, local reactive behaviors, and

Anchan, S., P, G., M, L., M, R., Y N, S. K. and B A, S.

Comparative Study of ORB-SLAM2 and IR-Based Revolution Counting with Ultrasonic Obstacle Avoidance for Autonomous Wheelchair Navigation.

DOI: 10.5220/0013575300004639

In Proceedings of the 2nd Inter national Conference on Intelligent and Sustainable Power and Energy Systems (ISPES 2024), pages 35-39

ISBN: 978-989-758-756-6

human inputs. The wheelchair is self-operating, that

is it goes to any given destination on its own. The

navigation system utilizes lasers to determine the

direction by locating certain landmarks. To detect

obstacles, IR proximity sensors are used and the

wheelchair will then bypass the obstacles using

reactive controllers. The wheelchair also has an

override control which allows the user to operate it

via a joystick and manually directs it to any other

point. Research by U. Masud. (Masud et al., 2017)

presents a vision-based control of a wheelchair,

allowing the users to have complete control of the

movement of the wheelchair using their eyes. This

allows users even with multiple disabilities to control

the wheelchair independently with no assistance from

anyone. As per research by Shahnaz et al (Mur-Artal

and Tardós, 2017). (2017) a low-cost smart electric

wheelchair that incorporates destination mapping and

intelligent control features. This study emphasizes the

importance of cost-effectiveness in design while

maintaining functionality. The wheelchair is

equipped with microcontrollers for autonomous

navigation, obstacle detection, and slope

management. The study demonstrates how smart

technology can be leveraged to improve wheelchair

accessibility and usability, particularly for users in

low-resource settings. there are plenty of ways to

make this wheelchair autonomous. In this paper, we

have used Orb_Slam2 (Oriented Fast and Rotated

Brief SLAM 2) which is a widely used open-source

simultaneous localization and mapping (SLAM)

system.

It enables real-time tracking, mapping, and

relocalization using monocular, stereo, or RGB-D

cameras. ORB-SLAM2 uses ORB features for

keypoint detection and description, making it

computationally efficient while maintaining accuracy.

In the paper (Mur-Artal et al., 2015) "ORB-SLAM2,"

the authors presented an improved SLAM system

capable of operating with monocular, stereo, and

RGB-D cameras. The system provides real-time

tracking, mapping, and relocalization functionalities,

leveraging ORB features for efficiency and

robustness. It introduces stereo and RGB-D support

while improving the loop-closing and relocalization

capabilities of its predecessor, ORB-SLAM. The

framework is versatile and suitable for robotics and

augmented reality applications, offering open-source

access for broader adoption and further research. In

the paper (Wolf, 2003) "ORB-SLAM," the authors

introduced a versatile and accurate SLAM system

designed for monocular cameras.

It provides real-time tracking, mapping, and

relocalization capabilities using ORB features for

efficiency. The system includes a robust loop-closing

mechanism to detect previously visited locations and

an effective map-recovery method for relocalization

after tracking loss. Its computational efficiency and

accuracy make it suitable for robotics and augmented

reality applications, and the open-source availability

facilitates broader adoption and innovation.

There are plenty of ways of controlling

wheelchairs other than joystick. Erik Jason Wolf

(Chieein et al., 2009) investigates the effects of

whole-body vibration on users of electric-powered

wheelchairs. The study evaluates wheelchair designs

and user exposure to vibrations, aiming to improve

comfort, safety, and health outcomes. The research

highlights the importance of biomechanical and

ergonomic considerations in wheelchair development

2 PROPOSED METHOD

Wheelchairs, when used for indoor navigation, have

to be very precise in identifying the objects around

them as an indoor setup might have multiple obstacles

at closer distances. So, we used orb-slam2 which

creates an indoor map of the surroundings with a

monocular camera in real time as navigation occurs,

and visual odometry for accurate motion within

intricate outdoor settings. This will improve mobility,

and assist users in overcoming difficult environments.

2.1 ORB SLAM 2

The autonomous navigation feature of the retrofittable

electric smart wheels utilizes ORB-SLAM2 (Oriented

FAST and Rotated BRIEF SLAM 2) for real-time

mapping and localization. ORB-SLAM2 uses ORB

features to track key points in the environment,

creating a map while maintaining accurate localization

even in dynamic settings. It is capable of loop closure

detection, which allows the wheelchair to recognize

previously visited locations and correct drift in its

map, making navigation more reliable. This system

enables autonomous movement without relying on

GPS, ideal for complex, outdoor environments.

2.2 Object Detection

Employing the use of InfraRed (IR) and ultrasonic

sensors: IR sensors detect objects by sending out an

IR wave from their transmitter and waiting for the

reflection back. These sensors are inexpensive and

assure good close-range object detection. Four IR

sensors were mounted on the wheelchair; two on the

handles and the other set near the wheels, to ensure

ISPES 2024 - International Conference on Intelligent and Sustainable Power and Energy Systems

that all the possible directions in which the

wheelchair is moving are scanned for any obstacles.

The signals from these sensors were however picked

up by the control unit to detect any obstruction in the

path of the wheelchair.

A drawback however of IR sensors is that they

cannot provide any information concerning the

distance of the detected object.

3 METHOD

3.1 SLAM Navigation with

ORB-SLAM 2

The system design for the customizable electric smart

wheelchair is detailed in Figure 3. The wheelchair is

mounted with a battery and a controller while the

computational activities are executed by a Jetson

Nano which runs the ORB-SLAM2 algorithm for

mapping and localizing the environment. There is a

wireless communication link between the Jetson and

the controller where the Jetson is used to control the

wheels' motion. A camera fixed on the Jetson device

provides the depth information that is transformed

into a point cloud. This form of data is paired with

Visual SLAM (VSLAM) techniques that allow the

wheels to navigate on their own without any usage of

GPS regardless of the mating surfaces.

Figure 1: Block diagram of SLAM-based navigation.

Figure 3. illustrates the block diagram of the

wheelchair system. The wheelchair is integrated with

a battery, BLDC motor, and motor controller that

drives the movement. The Jetson Nano serves as the

main control unit, responsible for processing data

from the camera and running the ORB-SLAM2

algorithm for autonomous navigation. A joystick is

connected to the Jetson for manual control, allowing

the user to override the autonomous system if needed.

Communication between the ESP32 motor

controllers is established via the ESP-NOW protocol,

providing efficient and low-latency wireless data

transfer for smooth coordination of the system

components

Figure 2: Block diagram of wheelchair.

3.2 Obstacle Avoidance Using IR

Sensor

The wheelchair comes with an ultrasonic sensor to

help identify obstacles and provide a safe way to

navigate. The sensor is always active, sweeping in

front of the wheelchair and perceptive to any objects

and barriers within the path. In case an obstacle is

sensed, the wheelchair automatically freezes and no

further motion of the machine is allowed, thus averting

all manner of accidents. The wheelchair stays in that

position until such a time the obstruction has been

removed or that a clear sunlit path is established and

then it continues moving without any delays.

The navigation system follows a set path that has

already been programmed. The wheelchair is

controlled, and the ultrasonic sensor scans for

obstacles as the wheelchair travels along the pre-

programmed path. This way, the wheelchair enables

safe and effective mobility indoors or within any

structured space making it comfortable for the users.

Ultrasonic sensors act as a safety measure for the

Figure 3: Working of the wheelchair with IR sensor.

Comparative Study of ORB-SLAM2 and IR-Based Revolution Counting with Ultrasonic Obstacle Avoidance for Autonomous Wheelchair

Navigation

system since they facilitate the detection and

avoidance of obstacles in real-time. This makes the

system applicable in environments where obstacles

may not be static and could come at any moment

4 RESULTS

In the paper, the authors use the KITTI dataset to

compare an IR- and ultrasonic-based system with a

stereo camera system in terms of timing and

navigation. Tracking-related tasks like ORB

extraction were slower using the stereo system than

compared to the IR and ultrasonic-based system,

~10.0 ± 5.0 ms compared to 24.83 ± 8.28 ms, pose

prediction ~3.0 ± 2.0 ms compared to 2.36 ± 1.84 ms

ascribed to the lesser number of features used, thus

lesser computational complexity. However, in the

case of mapping processes, that required slightly

more time: Local BA (~80.0 ± 40.0 ms vs. 69.29 ±

61.88 ms), and also Map Point Creation (~50.0 ± 20.0

ms vs. 47.69 ± 29.52 ms), since the involved working

with IR and ultrasonic data.

Table 1: Performance on the distance of navigation.

Performance on distance of navigation

Distance to be

navigated (m)

Navigation

accuracy

with ir (%)

Navigation

accuracy

with camera

(%)

> 10

5 to 10

2.5 to 5

2.5 to 1.5

Accuracy of navigation IR system The IR system

was less accurate at higher ranges (>10 m: 85% vs.

90%) but comparable for nearer ranges (2.5 to 5 m:

95%; 2.5 to 1.5 m: 98%). These results indicate that

though IR and ultrasonic systems can be excellent for

reasonable performance, the mapping capabilities and

long-range navigation accuracy are lagging behind

the stereo camera systems; it is very much important

to select the correct sensor system according to the

requirement of the application.

Table 2: Timing results comparison of each thread in ORB

SLAM 2 System to IR and Ultrasonic based System.

Settings

Dataset

KITTI

KITTI2

Sensor

Stereo

Camera

IR and

Ultra-

Sonic

Resolution

1226 × 370

Camera FPS

10 Hz

ORB

Features

2000

Tracking

ORB

Extraction

24.83 ±

8.28

~10.0 ± 5.0

Stereo

Matching

15.51 ±

4.12

Pose

Prediction

2.36 ± 1.84

~3.0 ± 2.0

Local Map

Tracking

5.38 ± 3.52

~6.0 ± 4.0

New

Keyframe

Decision

1.91 ± 1.06

~2.5 ± 1.0

Total Time

49.47 ±

12.10

~21.5 ± 9.0

Mapping

Keyframe

Insertion

~1.0 ± 0.5

Map Point

Culling

0.45 ± 0.38

~0.6 ± 0.4

Map Point

Creation

47.69 ±

29.52

~50.0 ±

20.0

Local BA

69.29 ±

61.88

~80.0 ±

40.0

Keyframe

Culling

0.99 ± 0.92

~1.5 ± 0.8

Total

129.52 ±

88.52

~133.0 ±

70.0

5 CONCLUSIONS

In conclusion, this research presents a promising

solution for enhancing wheelchair mobility. By

integrating autonomous navigation via ORB-SLAM,

the system offers both manual and autonomous

control to users, improving accessibility and

independence, particularly in outdoor environments.

The use of ESP32 communication for motor control

and depth-sensing cameras for navigation ensures

both reliability and precision. This adaptable system

not only addresses the limitations of existing

wheelchairs but also offers an affordable, scalable

solution for users in resource-constrained areas.

Future work will focus on refining the system’s

ISPES 2024 - International Conference on Intelligent and Sustainable Power and Energy Systems

robustness and expanding its capabilities for broader

applications.

REFERENCES

Sharafutdinov, D., Griguletskii, M., Kopanev, P., et al.

(2023). Comparison of modern open-source Visual

SLAM approaches. Journal of Intelligent & Robotic

Systems, 107(43). https://doi.org/10.1007/s10846-023-

01812-7

Merzlyakov, A., & Macenski, S. (n.d.). A comparison of

modern general-purpose Visual SLAM approaches.

Samsung Research.

Parikh, S. P., Grassi Jr., V., Kumar, V., & Okamoto Jr., J.

(2005). Usability study of a control framework for an

intelligent wheelchair. In Proceedings of the IEEE

International Conference on Robotics and Automation

(pp. 4745–4750). https://doi.org/1570853

Lim, A. R. G., Rabacca, J. C. D., Valdez, E., Sybingco, E.,

Sapang, O., Roque, M. A., & Gustilo, R. (n.d.). Wi-Fi

indoor navigation system controlled automated

wheelchair. Department of Electronics and

Communications Engineering, De La Salle University-

Manila.

Masud, U., Almolhis, N., Alhazmi, A., Ramakrishnan, J.,

Islam, F. U., & Farooqi, A. R. (2024). Smart wheelchair

controlled through a vision-based autonomous system.

IEEE Access, 12, 65099–65116.

https://doi.org/10.1109/ACCESS.2024.33956

Shahnaz, C., Maksud, A., Fattah, S. A., & Chowdhury,

S. S. (2017). Low-cost smart electric wheelchair with

destination mapping and intelligent control features. In

Proceedings of the IEEE International Symposium on

Technology and Society (ISTAS), Sydney, NSW,

Australia (pp. 1–6).

https://doi.org/10.1109/ISTAS.2017.8318978

Mur-Artal, R., & Tardós, J. D. (2017). ORB-SLAM2: An

open-source SLAM system for monocular, stereo, and

RGB-D cameras. IEEE Transactions on Robotics,

33(5), 1255–1262.

https://doi.org/10.1109/TRO.2017.2705103

Mur-Artal, R., Montiel, J. M. M., & Tardós, J. D. (2015).

ORB-SLAM: A versatile and accurate monocular

SLAM system. IEEE Transactions on

Robotics,31(5),1147–1163.

https://doi.org/10.1109/TRO.2015.2463671

Wolf, E. J. (2003). Evaluation of electric powered

wheelchairs and exposure to whole-body vibration

(Doctoral dissertation, University of Pittsburgh).

University of Pittsburgh ETD collection.

Cheein, F. A., De La Cruz, C., Carelli, R., & Bastos-Filho,

T. F. (2009). Solution to a door crossing problem for an

autonomous wheelchair. In Proceedings of the

IEEE/RSJ International Conference on Intelligent

Robots and Systems (pp. 4931–4936).

https://doi.org/10.1109/IROS.2009.5354292

Mandel, C., Luth, T., Laue, T., Rofer, T., Graser, A., &

Krieg-Bruckner, B. (2009). Navigating a smart

wheelchair with a brain-computer interface interpreting

steady-state visual evoked potentials. In Proceedings of

the IEEE International Conference on Robotics and

Automation (pp. 1118–1125).

https://doi.org/10.1109/ICRA.2009.5152429

Iturrate, I., Antelis, J., & Minguez, J. (2009). Synchronous

EEG brain-actuated wheelchair with automated

navigation. In Proceedings of the IEEE International

Conference on Robotics and Automation(pp.2318–

2325). https://doi.org/10.1109/ICRA.2009.5152679

Morales, Y., Kallakuri, N., Shinozawa, K., Miyashita, T., &

Hagita, N. (2013). Human-comfortable navigation for

an autonomous robotic wheelchair. In Proceedings of

the IEEE/RSJ International Conference on Intelligent

Robots and Systems (IROS 2013) (pp. 2737–2743).

https://doi.org/10.1109/IROS.2013.6696691

Tanaka, H., Sumi, Y., & Matsumoto, Y. (2012). A visual

marker for precise pose estimation based on lenticular

lenses. In Proceedings of the IEEE International

Conference on Robotics and Automation(pp.5222–

5227). https://doi.org/10.1109/ICRA.2012.6224736

Tanaka, H., Sumi, Y., & Matsumoto, Y. (2012). A high-

accuracy visual marker based on a microlens array. In

Proceedings of the IEEE/RSJ International Conference

on Intelligent Robots and Systems (pp. 4192–4197).

https://doi.org/10.1109/IROS.2012.6385917

Mur-Artal, R., & Tardós, J. D. (2017). ORB-SLAM2: An

open-source SLAM system for monocular, stereo, and

RGB-D cameras. IEEE Transactions on Robotics,

33(5), 1255–1262.

https://doi.org/10.1109/TRO.2017.2705103

Comparative Study of ORB-SLAM2 and IR-Based Revolution Counting with Ultrasonic Obstacle Avoidance for Autonomous Wheelchair

Navigation