A Flexible Scheduling Architecture of Resource Distribution Proposal for

Autonomous Driving Platforms

Hadi Askaripoor

, Sina Shafaei

and Alois Knoll

Informatics Department, Technical University of Munich, Boltzmannstr. 3, 85748 Garching bei M

unchen, Germany

Keywords:

Scheduling, Resource Planner, Autonomous Driving, Platform Architecture, Decisions Making.

Abstract:

Autonomous driving has attracted a signiﬁcant amount of attentions over the last ten years. Providing a ﬂexible

platform to schedule the executions of the tasks under hard real-time constraints is also a crucial matter which

needs to be taken into account by the integration of intelligent applications. In this work, we propose a resource

planner, consisting of a monitoring mechanism, context manager, and decision unit which facilitates the timing

requirements in the presence of AI-based applications for the autonomous vehicles.

1 INTRODUCTION

With the rise in the number of on-board sensor de-

vices, rapid changes in urban landmarks and trans-

portation facilities, and the complex road conditions

of people and vehicles, the ability to safely respond

to autonomous vehicles in real-time is constantly in-

creasing. The computing resource capabilities of on-

board hardware are limited, which present a serious

challenge to self-driving vehicles in real-time. The

resource requirements for information processing in

the vehicle’s automatic driving process are different

according to the applications. In the normal driving

scenario, the vehicle only needs to handle the corre-

sponding road conditions smoothly. However, in an

emergency scenario the processing time is as short

as a second, including the calculation of the emer-

gency measures and the time it takes to complete

the emergency measures. The computing resources

of self-driving cars are limited, and the transmission,

the brakes, and the steering modules in the car all re-

quire a reaction time. Furthermore, autonomous ve-

hicles employ plenty of safety-critical applications.

Therefore, if the scheduling strategy is not capable

of guaranteeing the intelligent system’s complete rea-

soning operations and responses to physical abnor-

malities within a certain time period, ending up in

hazardous situations is highly imminent— leading to

catastrophic consequences.

https://orcid.org/0000-0002-2570-5010

https://orcid.org/0000-0002-9381-0197

https://orcid.org/0000-0003-4840-076X

Currently, many useful and practical machine

learning-based applications exist, which are already

integrated in semi-autonomous vehicles and plat-

forms. For example, road image recognition, behav-

ior recognition, object recognition, decision making,

trajectory decision, and down-scaling techniques are

very popular applications among the developed solu-

tions to ease the decision making process for the in-

telligent vehicle. This represents the need to have a

coordinated scheduling mechanism for the resources

and the sensors combined with the highest level of

ﬂexibility that can work with the divers range of the

AI-based applications of the intelligent vehicle plat-

form.

2 RELATED WORK

Machine learning is the core of the AI-based appli-

cation in the intelligent vehicle and its applications

are distributed from the sensory level to the deci-

sion making level. A very intuitive example of this

process is found in the work of (Thammachantuek

et al., 2018a), in which they propose support vec-

tor machines (SVMs) for recognition of road images.

Four actions such as turn left, turn right, forward,

and stop are used as an example to verify the ac-

curacy of classiﬁcation results. For more complex

scenarios like having the pedestrian near the road,

the authors of (Fang et al., 2018) provide a solution

based on two sensors combining machine learning

techniques to monitor the motion of people. Further-

594

Askaripoor, H., Shafaei, S. and Knoll, A.

A Flexible Scheduling Architecture of Resource Distribution Proposal for Autonomous Driving Platforms.

DOI: 10.5220/0010471605940599

In Proceedings of the 7th International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS 2021), pages 594-599

ISBN: 978-989-758-513-5

Zonal Gateway

Resource Planner

Perception Module

Localization Module

Zonal Gateway ECU

Actutor

Sensor

Ethernet TSN Communication

Figure 1: Integration of the proposed resource planner in the platform of E/E architectures.

more, Hayat et al. (Hayat et al., 2018) takes it one

step further and proposes the use of multi-classes ob-

ject recognition by using CNN for this aim. Tham-

machantuek et al. (Thammachantuek et al., 2018b)

focus on decision-making and provide a performance

comparison of well-known supervised learning algo-

rithms such as SVM, MLP, CNN, DT, and RF. At the

end, the outcome of this application will be consumed

by other functions or applications like trajectory plan-

ning. Similar to the work of (Werling et al., 2010), a

trajectory generator may be proposed to generate dif-

ferent trajectories; however, machine learning tech-

niques will be employed on the ﬂy to choose the op-

timal trajectory. This example of dataﬂow from the

sensors to the applications demonstrates the potential

integration between machine learning solutions and

the importance of having a coordination mechanism

to distribute the available resources to desired entities

within the expected time frame.

Liu and Layland (Liu and Layland, 1973) pro-

posed the earliest deadline ﬁrst scheduling in their

classic work, which assumes that a multitasking sys-

tem consisting of several independent periodic tasks

runs on a single processor. The earliest deadline

ﬁrst algorithm is the optimal algorithm on the non-

mixed-critical system which can reach the maximum

resource utilization limit, however in a mixed-critical

system like autonomous vehicle it is unable to guar-

antee that emergency tasks will be executed ﬁrst. In

terms of ﬁxed-priority scheduling, the earliest dead-

line ﬁrst algorithm can ensure the high priority of

safety-critical tasks, but the lower-priority tasks can

wait an indeﬁnite amount of time before being exe-

cuted, leading to a higher task starvation and miss-

ing rate of low-priority tasks. Based on ﬁxed-priority

scheduling, an improved machine learning-based al-

gorithm as Fair Emergency First (FEF) is designed

for scheduling (Malik et al., 2019b). A scheduling al-

gorithm requires a framework or platform to be fully

integrated and also the onboard functions to be en-

abled. In this regard, FEF has been recently used as

the core approach of the hybrid agent in the scheduler

model of an autonomous vehicle (Malik et al., 2019a).

Kong and Wu (Kong and Wu, 2005) have proposed a

mechanism which gathers two inputs from the envi-

ronment for processing more dynamic scenarios.

Similarly, Yingzi et al. (Yingzi et al., 2009) pro-

posed a set of dispatching rules which are used to

schedule the task in dynamic task lists. The lists

convert the scheduling problem into a reinforcement

learning problem ﬁrst, followed by the construction

of a dynamic programming process. Shulga et al.

(Shulga et al., 2016) have aimed to extract the job in

sub-tasks and assign them as multiple threads to com-

putational units; therefore, a making-decision system

is placed ahead of the scheduling unit.

A Flexible Scheduling Architecture of Resource Distribution Proposal for Autonomous Driving Platforms

595

3 METHODOLOGY AND

PROPOSED ARCHITECTURE

In order to compute an architectural proposal based

on predeﬁned optimization goals by collecting system

hardware/software properties and applications with

a focus on artiﬁcial intelligence, we propose an ap-

proach in this work and refer to it as resource plan-

ner in the following sections. Our resource planner is

proposed for state-of-the-art electronic and electrical

(E/E) architecture of a vehicle as presented in Figure

1. The architecture of the resource planner consists of

three major mechanisms, namely Monitoring Mecha-

nism, Context Manager, and Decision Unit.

3.1 Monitoring Mechanism

To monitor application timing, we have designed a de-

centralized mechanism that focuses on identiﬁcation

of timing violations of each application deployed on

the E/E platforms of the vehicle. Since the automotive

applications are divided into the categories of safety-

critical and non-safety critical, the real-time monitor-

ing plays a pivotal role in precisely detecting timing

violations in safety-critical applications. These mech-

anisms have access to all nodes of the car topology,

where each node represents a speciﬁc application. In

our considered E/E architecture, as depicted in Figure

1, the monitoring approach is developed in the per-

ception and localization modules to monitor the tim-

ing of running applications for each one of those mod-

ules and the output is forwarded to the resource plan-

ner. As the input of the monitoring approach, timing

requirements related to each application (e.g., start-

ing times and deadlines) as well as the timing order

of each application (i.e., execution priority or paral-

lelism) are provided to the monitoring mechanism.

As the output, in the case of any missing applica-

tion deadline or timing violation, is exclusively imple-

mented on perception and localization modules, the

relevant timing violations are recognized and are for-

warded to a warning segment located in the decision

unit (See Fig. 2).

3.2 Context Manager

This mechanism aims to identify the driving context

of the vehicle. For instance, driving a car into a tun-

nel, in rainy weather conditions, and trafﬁc jam sit-

uation are interpreted as three driving situations. In

order to distinguish the driving context, an interaction

and data exchange must be done between the context

manager mechanism and other modules. More specif-

ically, the vehicle state (computed by the localization

module) as well as the objects around the vehicle (cal-

culated by the perception module) must be identiﬁed

simultaneously and fused into context manager in or-

der to compute the current driving context of the car.

Eventually, the result of this mechanism is utilized by

the decision unit. The labeling approach is used to

identify the driving situation in the context manager

with respect to localization and perception calcula-

tion. In other words, the vehicle state and the objects

around it are labeled periodically in such a way that

they make the situation of the vehicle distinguishable

according to its surrounding environments. There-

fore, in case of any updates in the labels, the resource

planner is notiﬁed accordingly.

Start

Applications Properties in Design Time: application deadline,

application output signal type (e.g., periodic or aperiodic), safety-

critical, or non-safety-critical type, etc.

Analysis and monitoring

of application timing

requirement during run-

time by monitoring

mechanism

Timing violation

Create a warning

and a proposal by

decision unit

Send the warning and

the proposal to the

violated application by

resource planner

End

Yes

Figure 2: The resource planner architecture to identify and

warn the timing violations in the proposed architecture.

3.3 Decision Unit

After recognition of the timing violations by the mon-

itoring mechanism, the result is sent to the warning

segment. In this segment, the violated module which

has missed the application deadline is identiﬁed (in

our scenario, the localization or perception module).

Accordingly, a warning regarding the relevant timing

violations generated by the decision unit is transferred

to the module by the resource planner. The module

is then informed about its timing violation, which is

relevant to a running application. Furthermore, a pro-

posal, including a suggested solution to decrease the

VEHITS 2021 - 7th International Conference on Vehicle Technology and Intelligent Transport Systems

596

Figure 3: The proposed architecture in ROS2.

timing violations in run-time, e.g., killing the appli-

cations in an optimized and safe manner, is created

by the decision unit. Accordingly, the module, which

has violated the timing requirement of the application,

can utilize the proposed solution to reduce the viola-

tion, while the functional safety point of view of the

applications and other modules must be considered at

run time (e.g., change the running order of the appli-

cations or killing the applications according to safety

aspects). In contrast to our previous work in progress

that the architecture synthesis was going to be per-

formed in the design-time (Askaripoor et al., 2020),

in this study the architecture synthesis is done in run

time focused on timing violations, as shown in Figure

As mentioned above, the result of the context

manager is forwarded to the decision unit as well. In

this unit, after labeling processing and understanding

the current context of the vehicle in real-time, aim is

to proceed with the reduction of the computational

power of the system. For example, when a car drives

into a tunnel, the Global Positioning System (GPS)

will fail; accordingly, detailed and accurate mapping

systems will help prevent accidents. Therefore, the

allocated resources to analyze the GPS data can be

assigned to other critical applications in the case of re-

source shortages. Another situation could be switch-

ing to lidar algorithms in rainy weather instead of

the camera because of the inefﬁciency of the camera

as the main source. This type of weather results in

killing camera-related algorithms concerning context

changes as well as functional safety aspects. Provid-

ing empty resources—in other words, decreasing the

computational power based on the context changes

as a proposal—is provided by the resource planner.

The created proposal is sent to the relevant modules;

whereas, its usage is only decided by them consider-

ing the functional safety aspects.

RP_Proposal Topic

Figure 4: The publish-subscribe pattern to communicate

with other modules.

We have utilized the publisher-subscriber pattern

to transmit the data between the resource planner and

the other two modules in our considered E/E archi-

tecture, as depicted in Figure 4. For instance, the re-

source planner can publish its proposal to the topic

channel and the localization module can subscribe to

this channel to access the proposal. Conversely, the

core information, including which application is run-

ning on which core, can be published to the topic

channel by a module (e.g., the perception module),

and the resource planner can subscribe to it in order

to acquire the status of the cores and calculate the pro-

posal, which was explained in decision unit, based on

the context manager and monitoring mechanism.

Furthermore, we have used an open-source Robot

A Flexible Scheduling Architecture of Resource Distribution Proposal for Autonomous Driving Platforms

597

Driver

Listener

Preprocessing Ground Filtering Clustering Shape Extraction

Resource Planner

Timing Analysis

Warning Proposal

Figure 5: Identifying the timing violation of the Lidar Object Detection Stack developed in Autoware by resource planner.

Operating System 2 (ROS2)(Thomas et al., 2014)

to implement our proposed publisher-subscriber pat-

tern considering the fact that it also supports real-

time communications. ROS2 employs Data Distri-

bution Service (DDS) as the backbone of the commu-

nications between publishers and subscribers. DDS

is suitable for real-time distributed embedded sys-

tems due to its different transport conﬁgurations

(e.g., fault-tolerance and deadline) and scalability

(Maruyama et al., 2016). The proposed architecture

utilized in ROS2 is presented in Figure 3.

4 EVALUATION CRITERIA AND

OUTLOOK

In order to evaluate our proposed approach, we have

deﬁned an automotive use case using Autoware (an

open-source ROS 1/2-based software for self-driving

vehicles)(Kato et al., 2018). Our developed approach

in ROS2 will connect to Autoware in such a way that

the data related to the localization and perception unit

ﬁt into the predeﬁned modules. In the next step, the

resource planner applies its mechanisms for the local-

ization and the perception units.

The topic statistics feature of ROS2 provides the

measurement of statistics for messages received by

any subscription, including message period, message

age, and data type value statistics (Thomas et al.,

2014). By using this method, monitoring the sub-

scription timing of each node related to a speciﬁc

application is feasible. However, there are some

limitations regarding this feature (e.g., enabling the

topic statistics only via the subscriber) which must

be solved by modifying the source code of the topic

statistics (e.g., the topic statistics enabled by using

ROS parameters).

Furthermore, using high-resolution clock in C++

programming language allows us to calculate a de-

sired node latency as well as a system latency (Strous-

trup, 2013). According to Figure 5, the node latency

(e.g., clustering) can be calculated by listening to the

time difference between its output and its input while

utilizing high-resolution clock. Similarly, the system

latency for the object detection stack can be com-

puted by subtracting the Shape Extraction node out-

put time and the Driver node input time utilizing the

same clock. In the next step, the timing analysis re-

garding the node or the system is transferred to the

resource planner. Subsequently, the timing violations

are identiﬁed, and a warning proposal is sent to the

nodes/system (here, object detection stack) which vi-

olate their timing requirements.

By following this proposed approach, we can

identify the possible timing violations in the system

as well as in the nodes based on our predeﬁned timing

requirements and notify the violated nodes or system.

5 CONCLUSION AND FUTURE

STEPS

In this paper, we presented a resource planning pro-

posal which not only identiﬁes the timing violations

that occurred in AI-based applications but also noti-

ﬁes the violated applications accordingly. Moreover,

the resource planner can recognize the driving con-

text of the vehicle by utilizing the pub/sub approach.

The proposed resource planner includes three meth-

ods such as monitoring mechanism, context manager,

and decision unit. In order to implement and evaluate

this approach, ROS2 and Autoware were speciﬁed to

be utilized respectively.

Our future work consists of the following steps,

implementing monitoring, warning mechanisms us-

ing Autoware stack to monitor system and node la-

tencies, measure message drops, and evaluate the ac-

curacy of the proposed warning mechanism.

VEHITS 2021 - 7th International Conference on Vehicle Technology and Intelligent Transport Systems

598

Secondly, integrating the resource distribution

mechanism, in the case of any core fails in a multicore

computational unit. In other words, the resource plan-

ner will recognize cores failure and then will assign

the safety-critical application, which was running on

the failed core, to the other non-safety-critical core,

which has a non-safety-critical application running.

Consequently, it will cause in distributing the con-

sumption of the cores and minimizing the message

drops, and timing latencies for safety-critical applica-

tions.

ACKNOWLEDGEMENTS

This work is part of the project ”KI-FLEX” (project

number 16ES1027) which is funded by German Fed-

eral Ministry of Education and Research (BMBF)

within the framework of the guidelines on promoting

research initiatives in the ﬁeld of “AI-based electronic

solutions for safe autonomous driving.

REFERENCES

Askaripoor, H., Farzaneh, M. H., and Knoll, A. (2020).

Considering safety requirements in design phase of

future e/e architectures. In 2020 25th IEEE Interna-

tional Conference on Emerging Technologies and Fac-

tory Automation (ETFA), volume 1, pages 1165–1168.

IEEE.

Fang, C.-C., Mou, T.-C., Sun, S.-W., and Chang, P.-

C. (2018). Machine-learning based ﬁtness behavior

recognition from camera and sensor modalities. In

2018 IEEE International Conference on Artiﬁcial In-

telligence and Virtual Reality (AIVR), pages 249–250.

IEEE.

Hayat, S., Kun, S., Tengtao, Z., Yu, Y., Tu, T., and Du,

Y. (2018). A deep learning framework using convo-

lutional neural network for multi-class object recog-

nition. In 2018 IEEE 3rd International Conference

on Image, Vision and Computing (ICIVC), pages 194–

198. IEEE.

Kato, S., Tokunaga, S., Maruyama, Y., Maeda, S.,

Hirabayashi, M., Kitsukawa, Y., Monrroy, A., Ando,

T., Fujii, Y., and Azumi, T. (2018). Autoware on

board: Enabling autonomous vehicles with embedded

systems. In 2018 ACM/IEEE 9th International Con-

ference on Cyber-Physical Systems (ICCPS), pages

287–296. IEEE.

Kong, L.-F. and Wu, J. (2005). Dynamic single machine

scheduling using q-learning agent. In 2005 Interna-

tional Conference on Machine Learning and Cyber-

netics, volume 5, pages 3237–3241. IEEE.

Liu, C. L. and Layland, J. W. (1973). Scheduling algo-

rithms for multiprogramming in a hard-real-time en-

vironment. Journal of the ACM (JACM), 20(1):46–61.

Malik, S., Ahmad, S., Kim, B. W., Park, D. H., and Kim,

D. (2019a). Hybrid inference based scheduling mech-

anism for efﬁcient real time task and resource man-

agement in smart cars for safe driving. Electronics,

8(3):344.

Malik, S., Ahmad, S., Ullah, I., Park, D. H., and Kim,

D. (2019b). An adaptive emergency ﬁrst intelli-

gent scheduling algorithm for efﬁcient task manage-

ment and scheduling in hybrid of hard real-time and

soft real-time embedded iot systems. Sustainability,

11(8):2192.

Maruyama, Y., Kato, S., and Azumi, T. (2016). Ex-

ploring the performance of ros2. In Proceedings of

the 13th International Conference on Embedded Soft-

ware, pages 1–10.

Shulga, D., Kapustin, A., Kozlov, A., Kozyrev, A., and

Rovnyagin, M. (2016). The scheduling based on ma-

chine learning for heterogeneous cpu/gpu systems. In

2016 IEEE NW Russia Young Researchers in Electri-

cal and Electronic Engineering Conference (EICon-

RusNW), pages 345–348. IEEE.

Stroustrup, B. (2013). The C++ programming language.

Pearson Education.

Thammachantuek, I., Kosolsombat, S., and Ketcham, M.

(2018a). Support vector machines for road images

recognition in autonomous car. In 2018 18th Interna-

tional Symposium on Communications and Informa-

tion Technologies (ISCIT), pages 291–293. IEEE.

Thammachantuek, I., Kosolsomnbat, S., and Ketcham, M.

(2018b). Comparison of machine learning algorithm’s

performance based on decision making in autonomous

car. In 2018 International Joint Symposium on Arti-

ﬁcial Intelligence and Natural Language Processing

(iSAI-NLP), pages 1–6. IEEE.

Thomas, D., Woodall, W., and Fernandez, E. (2014). Next-

generation ROS: Building on DDS. In ROSCon

Chicago 2014, Mountain View, CA. Open Robotics.

Werling, M., Ziegler, J., Kammel, S., and Thrun, S. (2010).

Optimal trajectory generation for dynamic street sce-

narios in a frenet frame. In 2010 IEEE International

Conference on Robotics and Automation, pages 987–

993. IEEE.

Yingzi, W., Xinli, J., Pingbo, H., and Kanfeng, G. (2009).

Pattern driven dynamic scheduling approach using re-

inforcement learning. In 2009 IEEE International

Conference on Automation and Logistics, pages 514–

519. IEEE.

A Flexible Scheduling Architecture of Resource Distribution Proposal for Autonomous Driving Platforms

599