AN INTELLIGENT MAINTENANCE BASED ON

MACHINE LEARNING APPROACH FOR

WIRELESS AND MOBILE SYSTEMS

Amine Chohra

, Felicita Di Giandomenico

, Stefano Porcarelli

and Andrea Bondavalli

Images Signals and Intelligent Systems Laboratory, Paris-East University, Avenue Pierre Point, 77127, Lieusaint, France

ISTI, National Research Center, 56124, Pisa, Italy

Computer Engineering Department, University of Pisa, 56126, Pisa, Italy

Systems and Informatics Department, University of Florence, 50134, Florence, Italy

Keywords: Complex and Uncertain Systems, Wireless and Mobile Communication Systems, Analysis and

Decision-Making, Deterministic and Stochastic Petri Nets, Database Audit Behaviors, Intelligent Software

Agent, Reinforcement Q-Learning and Supervised Gradient Back-Propagation Learning Paradigms,

Artificial Neural Networks, Optimal Maintenance Policies.

Abstract: To enhance wireless and mobile system dependability, audit operations are necessary, to periodically check

the database consistency and recover in case of data corruption. Consequently, how to tune the database

audit parameters and which operation order and frequency to apply becomes important aspects, to optimize

performance and satisfy a certain degree of Quality of Service, over system life-cycle. The aim of this work

is then to suggest an intelligent maintenance system based on reinforcement Q-Learning approach, built of a

given audit operation set and an audit manager, in order to maximize the performance (performability and

unreliability). For this purpose, a methodology, based on deterministic and stochastic Petri nets, to model

and analyze the dependability attributes of different scheduled audit strategies is first developed.

Afterwards, an intelligent (reinforcement Q-Learning) software agent approach is developed for planning

and learning to derive optimal maintenance policies adaptively dealing with the highly dynamic evolution of

the environmental conditions. This intelligent approach, is then implemented with feedforward artificial

neural networks under the supervised gradient back-propagation learning to guarantee the success even with

large state spaces, exploits intelligent behaviors traits (learning, adaptation, generalization, and robustness)

to derive optimal actions in different system states in order to achieve an intelligent maintenance system.

1 INTRODUCTION

The problem of protecting data, stored in internal

databases and used by a wide variety of systems

during their execution, against run-time corruption

has long been recognized to be a critical aspect

highly impacting on the overall system

reliability/availability. Wireless and mobile systems

are typical examples of systems whose activities rely

on databases, where the information is frequently

accessed and updated. Indeed, these systems heavily

rely on internal databases to keep trace of resource

usage status and user data in order to correctly

setting up and managing user calls. For this purpose,

databases are included, where data are organized in

such a way to capture the relationships existing

among them. Unfortunately, these databases are

subject to data corruption determined by a variety of

hardware and/or software faults, such as the huge

amount and the dynamic nature of data, internal

bugs and transient hardware faults. The occurrences

of such faults have the potential of yielding to

service unavailability with (possibly heavy)

consequences on Quality of Service (QoS). Thus, a

self-checking maintenance software, using data

audits with checks and recovery actions, has been

successfully developed achieving a high availability

in Lucent Technologies 5ESS switch system since

quite long time (Haugk et al, 1985). More recently, a

framework containing a database audit subsystem

has been developed in (Bagchi et al, 2001) for

database audit and control flow checking to maintain

the structural and semantic integrity of the database

and a preemptive control flow checking technique.

115

Chohra A., Di Giandomenico F., Porcarelli S. and Bondavalli A..

AN INTELLIGENT MAINTENANCE BASED ON MACHINE LEARNING APPROACH FOR WIRELESS AND MOBILE SYSTEMS.

DOI: 10.5220/0003611001150118

In Proceedings of the International Conference on Wireless Information Networks and Systems (WINSYS-2011), pages 115-118

ISBN: 978-989-8425-73-7

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

In this work, the authors claimed the need of both

database audit and control checking (mechanisms

for detecting errors in the data flow of the client) to

guarantee a high detection coverage. In general, off-

the-shelf database systems are equipped with

utilities to perform data audits, such as described in

(Haugk et al, 1985), (Costa et al, 2000), (Bagchi et

al, 2001), (Oracle8 Server Utilities).

Our research objective, in this work, is to

investigate on such approach to derive optimal

maintenance policies of database supports in such

systems. It is noteworthy then to underline the

specific characteristics possessed by such

communication systems which have to be taken into

account in devising approaches for their

maintenance. The two main factors characterizing

wireless communication systems are: 1) Short-

persistence of most of the data stored in the database

(typically, of the same duration of the user call). 2)

The highly dynamic evolution of the environmental

conditions (e.g., varying number of active calls) and

the changes over time of the requirements and

services offered from these communication systems.

These two factors make maintenance difficult to

achieve by traditional methods, and consequently

approaches using learning and adaptation to replace

missing or incorrect environment knowledge by the

experimentation, observation, prediction, and

generalization, come out to be very attractive.

The methodology using DEpendability

Evaluation of Multiple phased systems (DEEM) tool

to model and analyze the dependability attributes of

different scheduled audit strategies is developed.

This methodology, essentially based on

Deterministic and Stochastic Petri Nets (DSPN) and

supported by DEEM tool, aims to derive appropriate

settings for the order and frequencies of database

audits to optimize selected performance indicators.

Afterwards, an intelligent software agent based on a

reinforcement Q-Learning approach is developed for

planning and learning to derive optimal maintenance

policies adaptively and Artificial Neural Networks

(ANN) for its implementation.

2 INTELLIGENT

MAINTENANCE SYSTEM

Wireless and mobile systems include a database

subsystem, storing system-related as well as clients-

related information, and providing basic services to

the application process, such as read, write and

search operations. Data concerning the status, the

access rights and features available to the users,

routing information for dispatch calls, are all

examples of data contained in such database,

organized in appropriate data structures usually

called tables (e.g., database tables A, B, and C). The

database is subject to corruption determined by a

variety of hardware and/or software faults, such as

internal bugs and transient hardware faults. The

occurrences of such faults have the potential of

yielding to service unavailability. Because of the

central role played by such database in ensuring a

correct service to clients, means to pursue the

integrity/correctness of data have to be carried out.

The synopsis, shown in Figure 1, of an intelligent

database maintenance system, built of a given audit

operation set, and an audit manager, is suggested in

order to allow to select, in each time period, the

optimal maintenance policy, the optimal audit

behavior. The part, in Figure 1, labelled "Relevant

Parameters" indicates those parameters of the

wireless communication systems which determine

the states space of these systems, mainly the time

(the nature of the application under study imposes

the time as relevant parameter), the mean number of

user calls N

call

, and the pointer failure rate λ

2.1 Audit Operation Set

In this work, we are not interested in defining or

analyzing audit operations from the point of view of

the detection and/or correction capabilities offered

by them. Instead, a given set of audit operations is

assumed to be provided (as shown in Figure 1 e.g.,

Audit1_AB, Audit1_BC, Audit2_AB, Audit2_BC

are audit operations dealing with database tables A,

B, and C) to cope with data corruption, where each

audit operation is characterized by a cost (in

execution time) and coverage (as a measure of its

ability to detect and/or correct wrong data).

2.2 Audit Manager (Decision-making)

The audit manager is responsible for applying a

maintenance strategy to cope with database

corruption and therefore preventing system

unavailability; it activates different audit operations

at different time intervals. To achieve this goal, it

has to select the part of the database to

check/recover, the detection/recovery scheme to

apply, and the frequency with which each

check/recovery operation has to be performed. It is

implemented by a decision-making subsystem which

integrates a methodology to model and analyze

maintenance strategies (where e.g., Table Pointers

are

structured in homogeneous sets A, B, and C as

WINSYS 2011 - International Conference on Wireless Information Networks and Systems

116

Audit Operation

Set

e.g., Audit1_AB,

Audit1_BC,

Audit2_AB,

Audit2_BC.

Wireless Communication System

Intelligent Database Maintenance System

Relevant Parameters

e.g., time, N

Call

Audit Manager

Intelligent Software Agent:

- Reinforcement Q-Learning,

Neural Networks.

Methodology Using DEEM Tool:

- DSPN Models,

e.g., Table Pointers Structured in

Homogeneous Sets

A, B, and C.

- Dependability Analysis,

- Planning.

Database

e.g., Database Tables

A, B, and C.

Figure 1: Synopsis of an intelligent maintenance system.

shown in Figure 1) and an intelligent software agent.

Current database maintenance systems are facing

with less predictable and more complex

environments such as the dynamic environment of

wireless communication systems. For more QoS,

database maintenance approaches must drop the

assumption that perfect environment knowledge is

available. They have to explore, check, recover from

failures autonomously their environments in real-

time (Kemme et al, 2011). More, if database

maintenance systems lack knowledge about

themselves and their environments, learning and

adaptation become then inevitable to replace

missing or incorrect environment knowledge by the

experimentation, observation, prediction, and

generalization. Indeed, the interest in complete

(intelligent) database maintenance systems dealing

with highly dynamic environments is becoming

necessary from embodied cognitive science and

understanding natural intelligence perspectives

(Pfeifer and Scheier, 1999), (Chohra, 2001).

3 MODELING AND ANALYSIS

OF DATABASE

MAINTENANCE

In this phase, our goal is to identify a methodology

to model and evaluate the relevant dependability

attributes of scheduled audit strategies in order to

derive optimal maintenance solutions for a given

setting of the system and environment parameters: 1)

the representation of the basic elements of the

system and the ways to achieve composition of

them; 2) the behavior of the system components

under fault conditions and under audit operations to

restore a correct state; 3) the representation of failure

conditions for the entire system; 4) the interleaving

of audits with on-going applications and their

relationships; 5) the effects of (combinations of)

basic audit operations on relevant indicators for the

system performance, in accordance with applications

requirements. Our approach is based on

Deterministic and Stochastic Petri Nets (DSPN).

Specifically, we defined general models which

capture the behavior of the database and of the

maintenance policy checking/recovering it, to be

easily adapted to the implementation of database

audits. This allows then analyzes which give a useful

indication about the tuning of the major parameters

involved in the database audit. The optimal trade-off

between the audit frequency and the investment to

improve the coverage of the audits can be found, to

match the best performability and dependability

constraints.

4 TOWARDS OPTIMAL

MAINTENANCE

In this Section, a reinforcement Q-Learning

approach is suggested for a dynamic (adaptive)

selection of the maintenance policy at varying

database and environmental parameter values

leading to select, at each time period, the optimal

maintenance policy, the optimal audit behavior.

Indeed, this adaptive selection is very important and

necessary to carry out the changes which could

occur during the life-cycle system which are not

foreseeable and react consequently by reinforcement

learning during on-line learning. While the

foreseeable changes are carried out using DEEM

tool as environment model for planning during off-

line learning. Thus, to dynamically adapt

maintenance policies at varying database and

environmental parameter values, reinforcement Q-

Learning approach, in Figure 2, is suggested.

5 ANN IMPLEMENTATION

The utility (performance) network having multiple

ANN under supervised gradient back-propagation

learning with single output, illustrated in Figure 3, is

suggested to implement the reinforcement Q-

Learning approach.

AN INTELLIGENT MAINTENANCE BASED ON MACHINE LEARNING APPROACH FOR WIRELESS AND

MOBILE SYSTEMS

117

Wireless Communication System (Real Environment)

(time, N

Call

, λ

)

Data

Transfer

time

period

2*tim

periods

3*time

periods

k*time

periods

Complete

Chec

Complete

Check

Optimal Audit Behaviors

Max(Q[S][A])

Intelligent Software Agent

(Reinforcement Q-Learning)

DEEM Tool

(Evaluating Environment Model)

action

state

reward

Performance

(Performability

and Unreliability)

t+1

t+ 1

Artificial Neural Networks

(Q[S][A])

Updating

Data Transfer

time

Relev ant Parameter s Databas e

Audit

Manager

Audit

Operation Set

Optimal

Maintenance Policies

(Opt imal Audit Behaviors)

Figure 2: Q-Learning approach synopsis.

6 CONCLUSIONS

From a given audit operation set, maintenance

database approaches aim to make in use an efficient

decision-making system to schedule these audit

operations, which defines their order and frequencies

in order to maximize the performance of a

considered wireless communication system. The

interest is then focalized on the database

maintenance approaches devoted to how to

optimally schedule a given audit operations and

particularly in which order and frequencies. For this

purpose, two audit optimization strategies based on

statistics collected during system operation have

been developed in (Bagchi et al, 2001) while we

suggest in this work a reinforcement Q-Learning

approach, i.e., audit behavior learning, implemented

by ANN under supervised GBP learning. This

approach exploits essential traits of intelligent

behaviors such as the learning ability of Q-Learning

paradigm to converge to the optimal actions and the

generalization and robustness capabilities of ANN in

order to achieve an intelligent database maintenance

system. Then, one important issue to guarantee the

success of the reinforcement Q-Learning approach is

developed suggesting ANN implementation to

derive an efficient state space representation.

An interesting alternative, for future research, is

to extend the number of relevant actions

considering: different time durations of the

operational phases, different audit operations, and

different combinations of the operational phases and

audit operations.

with i = 0, 1 ; k = 0, ..., K ; and j = 0.

Output Layer

Hidden Layer

Input Layer

action_0

time

Ca ll

Output Layer

Hidden Layer

Input Layer

action_3

time

Ca ll

Figure 3: Feedforward ANN to implement the

reinforcement Q-Learning.

REFERENCES

Adgar, A., Addison, J. F. D., and Yau, C.-Y., 2007.

Applications of RFID technology in maintenance

systems. 2nd World Congress on Engineering Asset

Management, Harrogate, UK.

Bagchi, S., Liu, Y., Whisnant, K., Kalbarczyk, Z., Iyer, R.,

Levendel, Y., and Votta, L., 2001. A framework for

database audit and control flow checking for a wireless

telephone network controller. Int. Conf. on

Dependable Systems and Networks, pp. 225-234,

Göteborg, Sweden.

Chohra, A., 2001. Embodied cognitive science, intelligent

behavior control, machine learning, soft computing,

and FPGA integration: Towards fast, cooperative and

adversarial robot team (RoboCup), Technical GMD

(currently Fraunhofer) Report, No. 136, pp. 01-59,

Sankt Augustin, Germany.

Costa, D., Rilho, T., and Madeira, H., 2000. Joint

evaluation of performance and robustness of a COTS

DBMS through fault-injection. Int. IEEE Conf. on

Dependable Systems and Networks, pp. 251-260, New

York, USA.

Haugk, G., Lax, F. M., Royer, R. D., Williams, J. R.,

1985. The 5ESS switching system, maintenance

capabilities. AT&T Technical Journal, Vol. 64, No. 6,

pp. 1385-1416.

Irigaray, A. A., Gilabert, E., Jantunen, E., Adgar, A.,

2009. Ubiquitous computing for dynamic condition-

based maintenance. Journal of Quality in Maintenance

Engineering, Vol. 15 Iss: 2, pp.151 – 166.

Pfeifer, R., and Scheier, C., 1999. Understanding

intelligence, MIT press.

WINSYS 2011 - International Conference on Wireless Information Networks and Systems

118