A Stochastic Queueing Model for Multi-robot Task Allocation

Ang´elica Mu˜noz-Mel´endez

1,2

, Pritviraj Dasgupta

and William Lenagh

Computer Science Department, National Institute of Astrophysics, Optics, and Electronics,

Luis Enrique Erro 1, Sta Ma Tonantzintla, Puebla 72840, Mexico

Computer Science Department, University of Nebraska at Omaha, Omaha NE 68182, U.S.A.

Keywords:

Robot Team, Task Allocation, Demand, Stochastic Queue.

Abstract:

A central problem in multi-robot systems is to solve the multi-robot task allocation problem. In this paper, a

decentralized stochastic model based on stochastic queueing processes is applied for an application of collec-

tive detection of underground landmines where the robots are not told the distribution or number of landmines

to be encountered in the environment. Repeat demands of inspection in the environment to ensure the accu-

racy of robot ﬁndings are necessary in this application. The proposed model is based on the estimation of a

stochastic queue of pending demands that represents the alternatives of action for a robot and is used to nego-

tiate possible conﬂicts with other robots. We compare and contrast this method with a decentralized greedy

approach based on the distance towards the sites where inspection demands are required. Experimental results

obtained using simulated robots in the Webots

 environment are presented. The performance of robots is

measured in terms of two metrics, completion time and distance traveled for processing a demand. Robots

applying the stochastic queueing model obtained competitive results.

1 INTRODUCTION

Over the past few years, multi-robot systems have

been successfully employed to solve problems in

several robotic domains such as unmanned search

and rescue, autonomous exploration of regions that

are difﬁcult for humans to maneuver in, automated

surveillance and security, automated agriculture and

domestic applications, etc. In each of these appli-

cations, the operations performed by the robots to

achieve the desired objective are abstracted as tasks.

A central problem in multi-robot systems is to solve

the multi-robot task allocation problem (MRTA) -

how to ﬁnd a suitable assignment of tasks to robots

so that the tasks performed by the robots can be com-

pleted in an efﬁcient manner in terms of time and

energy expended by the robots. We consider a cat-

egory of MRTA problems called ST-MR-TA (single

task robot, multi-robot tasks, time extended assign-

ment) (Gerkey and Matari´c, 2004), where ST stands

for single-task robots, i.e., each robot is able to ex-

ecute as most one task at a time, MR means multi-

robot tasks, tasks that require multiple robots to be

completed, and TA means time-extended assignment,

problems where the information to allocate tasks to

robots arrives over time. As a motivating application

of such an MRTA problem, we consider a multi-robot

landmine detection scenario. A task in this scenario

corresponds to a certain number of robots visiting the

location of a potential landmine, not necessarily at the

same time, to analyze the object using the robots’ sen-

sors. The location of potential landmines arrives dy-

namically and is made available to the robots. MRTA

in such a scenario correspondsto the multi-city travel-

ing salesman problem that has been shown to be NP-

hard (Miller et al., 2006) (Dasgupta, 2011). Previ-

ous work in MRTA for ST-MR-TA problem consid-

ers local or market-based heuristics. In this paper,

we propose to use a stochastic model called stochastic

queueing to address the MRTA problem (Huang and

Serfozo, 1999). Using spatial queueing is attractive

for our ST-MR-TA MRTA problem as it provides a

formal framework for distributed decision making by

the robots so that they can respond efﬁciently to dy-

namic changes in the task distribution. We compare

the performance of the spatial queueing MRTA algo-

rithm with a greedy MRTA algorithm based on the

distance to reach tasks. We have validated our algo-

rithm on the Webots simulator using a wheeled robot

called Corobot for different numbers of robots and

tasks. Our experimental results done with Corobot

robots on the Webots

 simulator for spatially dis-

tributed task allocation scenario show that teams of 5

and 10 robots using the stochastic queuing algorithm

256

Muñoz-Meléndez A., Dasgupta P. and Lenagh W..

A Stochastic Queueing Model for Multi-robot Task Allocation.

DOI: 10.5220/0004049102560261

In Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2012), pages 256-261

ISBN: 978-989-8565-21-1

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

perform less useless movements that result in shorter

traveled distances and completion time compared to

the robots using the greedy algorithm.

2 RELATED WORK

The problem of MRTA has been investigated us-

ing different techniques (Gerkey and Matari´c, 2004;

Mataric et al., 2003), and, recently with market-based

approaches (Dias et al., 2006). One of the earliest sys-

tems using for MRTA was the M+ system (Botelho

and Alami, 1999). In (Gerkey and Matari´c, 2004)

a widely accepted taxonomy for MRTA problems is

provided. The problems are classiﬁed along three

dimensions: (a) single task robots (ST) vs. multi-

task (MT) robots, related to the parallel task perform-

ing capabilities of robots, (b) single robot (SR) task

versus multi-robot (MR) tasks, related to the num-

ber of robots required to perform a task, and, (c) in-

stantaneous assignment (IA) versus time extended as-

signment (TA), related to the planning performed by

robots to allocate tasks. Mataric et al. compare per-

formance of robots teams using auction-based strate-

gies for coordination and commitment and report that

the least time is required to complete all tasks (put

out all alarms) when the robots are allowed to coor-

dinate their plans with each other as well as to dy-

namically change their plans (Mataric et al., 2003).

The traderbots approach by (Dias, 2004) uses multi-

round, single-item auctions for dynamic task alloca-

tion across multiple robots, while in (Jones et al.,

2006) the traderbots approach is augmented using

the Skill, Tactics, Play (STP) approach for coordi-

nated teamwork. The MRTA problem has also been

approached as an exploration problem of matching

a set of robots to a set of targets using an algo-

rithm called PRIM-ALLOCATION (Lagoudakis et al.,

2005). Zlot and his team havealso used auction-based

algorithms for multi-robot task allocation (Zlot, 2006;

Jones et al., 2011). The MRTA problem has also been

combined with techniques from multi-agent coordi-

nation and optimization techniques such as negotia-

tion (Viguria et al., 2007), coalition formation (Tang

and Parker, 2007), reinforcement learning (Schneider

et al., 2005), vector regression learning (Jones et al.,

2006), Hungarian algorithm (Liu and Shell, 2011),

vacancy chains (Dahl et al., 2009), and dynamic vehi-

cle routing (Bullo et al., 2011) to improve the perfor-

mance of the robots and deal with uncertainty.

3 PROBLEM FORMULATION

Our MRTA model is based on spatial queueing the-

ory. Probability and queueing approaches provide an

attractive formalism to model settings where multi-

ple targets have to be kept under surveillance by mul-

tiple mobile units in applications such as automated

surveillance, automated reconnaissance, etc. In the

queueing model, the requirement of an operation by a

robot on or at a target is referred to as a demand corre-

sponding to that target. In our model, the demands at

different targets are generated stochastically depend-

ing on the availability of target. A solution to the

problem consists of each robot calculating an ordered

sequence of demands based on the costs to process de-

mands while minimizing certain metrics such as the

distance traveled by the robots to process demands,

or, the waiting/idle time for targets. When the spatial

distribution of demands(targets) in the environment

is known, a queueing approach can be enriched by

applying a spatial framework. These systems gener-

ally evolve over time as Markovian processes and the

robots select tasks according to a Markovian mecha-

nism.

Let E ⊂ R

represent a bounded 2-D environment

and R = {r

: 1 ≤ i ≤ m} represent a set of m mo-

bile robots that are deployed within E. p

(t) ∈ E de-

notes the position of robot r

at time t. There are n

stationary targets distributed within the environment.

Each target requires a subset of robots in R to oper-

ate upon it. The set of operations performed by dif-

ferent robots on a target is referred to as a task. Let

T = {τ

: 1 ≤ i ≤ n} represent a set of tasks. Each

task τ

is associated with four attributes: its position

in the environment p

∈ E, a demand value nd

∈ Z

that denotes the number of robots that need to operate

on the task to complete it, a progress value ad

∈ Z

that denotes the number of robots that have already

serviced the task, and, a Boolean availability value

avail

denoting whether the task is currently being

serviced by a robot and is consequently unavailable.

Let T

open

= {τ

∈ T : ad

< nd

}, T

closed

= T \ T

open

and, T

avail

= {τ

∈ T

open

:avail

= true} represent the

sets of open, closed and available tasks respectively.

=|| p

− p

|| is the Euclidean distance between

tasks τ

and τ

and

=|| p

(t) − p

|| is the Eu-

clidean distance between robot r

at time t and task

. When a robot reaches the location of a task, one

unit of the task’s demand is processed and its progress

increases by 1. A task is completed when its progress

matches its demand, i.e., when ad

= nd

We represent the probability of a robot to select

task τ

after it has serviced task τ

as an inter-task

transition matrix M

given by:

AStochasticQueueingModelforMulti-robotTaskAllocation

257







... π

...

... π







, (1)

where π

∑

j6=i

is the inverse of the Euclidean

distance between tasks τ

and τ

normalized over all

tasks. Note that

∑

= 1. Also, since a robot needs

to service a task’s demand at most once, π

= 0.

Using this inter-task transition matrix, a robot has a

higher likelihood of selecting available tasks that are

closer than those that are further away.

The problem facing robot r

is to select a task τ us-

ing the probabilities in the inter-task transition matrix.

However, selecting the probabilities from M

does not

incorporate the dynamic nature of the system man-

ifested through robots servicing and accomplishing

tasks. Therefore, each robot r

maintains a local copy

of M

denoted by M

τ,r

and updates it using its own

task servicing information. When a robot services a

demand of task τ

, it sets the column j in M

τ,r

corre-

sponding to task τ

to zero, to indicate that it will not

service demands from task τ

in the future and con-

sequently, not include τ

in making its decisions. To

select tasks, each robot r

represents its probability of

selecting a task at time t as a vector state matrix V

robot r

’s initial vector state, V

(0), is given by:

(0) =



...



, (2)

where

∑

j6=i

is the inverse of the Euclidean dis-

tance between robot r

’s initial location and the loca-

tion of task τ

, normalized over the distances of r

all tasks.

is updated using the equation:

(t) = V

(t − 1) × M

τ,r

(t) (3)

Robot r

makes a decision about the next task to

process by selecting a next task according to the high-

est probability of tasks in V

(t). Since the robots se-

lect tasks in a distributed manner, more than one robot

end up selecting the same task. In that case, the task

is allocated to the robot with the higher probability

of performing the task. If more than one robot have

the same probability, the task is allocated to the robot

with the highest identiﬁer.

From the probabilities in V

(t) a robot can order

a queue of open tasks at time t, Q

(t) ⊆ V

(t), ac-

cording to the probabilities of the tasks. The values

of Q

, corresponding to the original values

sorted

in descending order, represent the set of current alter-

natives for robot r

at time t, as expressed in Equation

4. Note that q

∈ Q

represents the ﬁrst option for

robot r

. The stochastic queue represents, at the same

time, a negotiation tool to apply in case of conﬂicts

generated in task selection by other robots.

(t) =



,. .., q

n(t)r

: q

≥ q

j+1 r

∧

∈ T

open



(4)

4 STOCHASTIC QUEUEING

BASED ALGORITHM FOR

MRTA

The algorithm provides robots a mechanism for sort-

ing prospective tasks according to the available knowl-

edge. Prospective tasks are arranged in a queue of

probabilities Q

(t) that represents the list of preferred

tasks to be serviced by robot r

as described in Section

3. The procedure applied for robots to allocate tasks

demands is summarized in Algorithms 1 and 2.

The environment is explored applying transition

matrix M

and vector states of the robot V

(0) and

(t) calculated from the spatial distribution of tasks

and location of robot, as expressed in Eqs. (1) - (2),

see lines 3-16 of Algorithm 1.

Figure 1 illustrates the graph based on the transition

probabilities calculated using an environment compris-

ing 6 tasks.

5 EXPERIMENTAL RESULTS

The stochastic queueing based algorithm was com-

pared with a greedy algorithm under identical scenar-

ios of tasks service. In our greedy approach incom-

ing requests of service are released by a call center to

the robot team, and tasks are allocated to the nearest

available robot, on a ﬁrst-come ﬁrst-served basis. The

greedy algorithm allocates tasks to robots based on the

distance between robots and demands as the ﬁrst crite-

rion, and as a second criterion in the numerical identi-

ﬁer of robots to solve eventual conﬂicts that cannot be

solved by the former. In the stochastic approach robots

receive a copy of the list of tasks and estimate locally

the transition probabilities, vector state and queue of

lists to service.

Since this research focuses on task allocation we do

not deal with localization or path planning issues. We

assume a robot is able to reach the point in the environ-

ment where a demand of service is active.

We conducted a set of experiments using simulated

robots in the Webots

 environment. We used three

robot teams with 5 and 10 robots, with a varying num-

ICINCO2012-9thInternationalConferenceonInformaticsinControl,AutomationandRobotics

258

Algorithm 1: Stochastic Queueing based Task Allocation.

Input: set of tasks, T; current position of the robot, p

Local variables: list of open tasks, T

open

; list of avail-

able tasks T

avail

; list of non inspected tasks for robot r

non−insp

; transition probability matrix, M

; vector state of

the robot, V

(t); subsequent vector state of the robot,V

(t+

1); list of open available tasks selected by robot r

, Qt

(t);

list of probabilities of tasks in Qt

, qp

; boolean result of an

allocation, talloc.

1: begin

2: Initialize τ

.ad ← 0 ∀τ

∈ T.

3: Calculate T

open

← {τ

} : τ

.ad < τ

.nd ∀τ

∈ T

4: Calculate T

avail

← {τ

} : p

6= p

∀τ

∈ T

open

∧ ∀r

∈

5: Calculate T

non−insp

← {τ

} : τ

.ad = 0 ∀τ

∈ T

open

6: Calculate the transition matrix M

from T, applying Eq.

(1).

7: Calculate the initial state vector of robot r

(0), using

, coordinates of tasks in T, and the next state vector

(1), applying Eqs. (2) and (3).

8: Build a queue of preselected tasks Q

(t) with probabil-

ities qp

from V

(1), applying Eq. (4).

9: while T

non−insp

0 do

10: talloc ← SelectOneTask(T

open

,qt

,qp

11: if talloc = true then

12: Remove the serviced tasks from the local copy of

T and recalculate T

13: Update the set of current open tasks T

open

14: Update V

(t) and V

(t + 1) considering the cur-

rent position of the robot and applying Eqs. (2)

and (3).

15: Build a queue of preselected tasks Q

(t) with

probabilities qp

from V

(t + 1), applying Eq.

(4).

16: end if

17: end while

18: end

ber of tasks from 6, 12, 18 and 24. The robot model

used in these experiments is based on the Coroware

CoroBot robot, illustrated in Figure 2. It is equipped

with a four-wheel drive base and four infra-red sen-

sors, two located in the front and two on the sides of the

robot, for avoiding collisions, and a GPS for localiza-

tion. The maximum speed at which a robotcan travel is

0.8m/s. The robots are able to communicate wirelessly

within the environment that measures 20m× 20m. We

assume reliable communication.

The simulations were run on Webots 6.3.0 on Win-

dows 7. The location of tasks and robots were gen-

erated randomly applying a uniform distribution. The

number of demands for each task is a random num-

ber between 3 and 5. Since each task represents an

inspection site for ground landmines, one task cannot

be serviced twice by the same robot. The problem is

considered solved if all the tasks have been inspected

the associated number of demands.

We compare the performance of robot teams rely-

ing on both methods, greedy and stochastic in terms of

two metrics: completion time and distance. The men-

Algorithm 2: SelectOneTask (T,T

open

,qt

,qp

Input: set of tasks; T; set of open tasks, T

open

; list of open

tasks selected by robot r

, qt

; list of probabilities of tasks

in qt

, qp

Output: boolean value, talloc.

Global variables: set of robots R.

Local variables: ﬂoat: max qp

; list of robots: Ω.

Functions: max(x

,... , x

) returns the maximum

value of the set {x

,... , x

}; id(r

) returns i.

1: begin

2: Select the ﬁrst open task from qt

, qt

3: Broadcast qt

and qp

4: if qt

6= qt

,∀r

∈ R : r

6= r

then

5: qt

is allocated to robot r

6: talloc ← true (there are no conﬂicts)

7: else

8: if qp

> qp

∀r

∈ R : r

6= r

∧ qt

= qt

then

9: qt

is allocated to robot r

10: talloc ← true (r

has the highest probability to

reach the task).

11: else

12: max qp

← qp

: qp

> qp

∧qt

= qt

∀r

∈ R.

13: if qp

= max p

then

14: Ω ← {r

} ∀r

∈ R : r

6= r

∧ qt

1r j

= qt

1ri

∧

= max qp

15: if Ω 6=

0 then

16: if id(r

) = max(id(r

),... , id(r

)) : r

∈ Ω

then

17: qt

is allocated to robot r

, the robot

with the highest id.

18: else

19: talloc ← f alse (qt

is allocated to a

robot in Ω with an id higher than id(r

)).

20: end if

21: else

22: talloc ← false (r

/∈ Ω, qt

is allocated to

a robot with better probability to reach it).

23: end if

24: else

25: talloc ← false (qt

is allocated to a robot

with better probability to reach it).

26: end if

27: end if

28: end if

29: if talloc = true then

30: Update state of allocated task qt

in T and broad-

cast information.

31: Service task qt

and update qt

.ad in T.

32: Remove qt

and qp

from qt

and qp

33: Update state of qt

T, and content of T

open

and

broadcast information.

34: else

35: Remove qt

and qp

from qt

and qp

36: end if

37: return talloc.

38: end

tioned metrics were averaged across 5 runs of each

set of settings. Environments are selected from a set

AStochasticQueueingModelforMulti-robotTaskAllocation

259

5 10 15 20

τ6

τ1

τ2

τ3

τ4

τ5

Tasks

(a)

0.17

0.24

0.26

0.12

0.21

0.11

0.07

0.22

0.09

0.52

0.29

0.15

0.22

0.19

0.16

0.18

0.25

0.13

0.30

0.17

0.19

0.21

0.25

0.18

0.12

0.48

0.07

0.25

0.08

(b)

Figure 1: Example of environment containing 6 tasks, the

values on the axes represent meters (a), and corresponding

transition matrix (b) based on the extrapolation of inverse

distances between pairs of tasks.

Figure 2: Corobot robot simulated in the environment We-

bots.

of 30 environments previously generated. Algorithms

are tested under identical conditions.

Completion time is the time in the simulation

when all the tasks are completed. This metric, plot-

ted in Figure 3, indicates how long it takes to the

robot team inspecting the environment. Note that, in

general, robots using the stochastic algorithm invest

less time to completedemands of inspection than their

partners using the greedy algorithm.

The distance traveled by robots to service tasks is

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

6 12 18 24

Time (ticks)

Number of Tasks

Average Completion Time

GREEDY: 5 robots

10 robots

STOCHASTIC: 5 robots

10 robots

Figure 3: Average Completion Time.

100

120

140

160

180

200

220

6 12 18 24

Distance (m)

Number of Tasks

Average Distance

GREEDY: 5 robots

10 robots

STOCHASTIC: 5 robots

10 robots

Figure 4: Average Distance.

also considered in this comparison, and it is shown

in Figure 4. This plot shows how the time invested

by robots when selecting tasks has a positive effect

in their decisions, and that the greedy approach can

face drawbacks when dealing with complex environ-

ments in terms of number of robots and tasks. Re-

garding the distance the robots using the stochastic

algorithms travel up to 50% less that their partners

using the greedy algorithm, and that happens again in

the scenarios using 18 and 24 tasks.

6 CONCLUSIONS AND

PERSPECTIVES

One stochastic queueing based algorithm for task al-

location for robot teams is presented and compared

with a simple greedy algorithm in terms of comple-

tion time and distance. These algorithms have been

tested in a scenario of landmine detection where the

inspection points are modeled as tasks that are ser-

viced by different robots a number of demands.

In the near future, we plan to conduct experi-

ments in dynamic scenarios where information of

ICINCO2012-9thInternationalConferenceonInformaticsinControl,AutomationandRobotics

260

some tasks is available a priori, but also information

of new tasks arrives on-line, and is shared and in-

tegrated by robots in their decision making process.

The stochastic queueing algorithm already recalcu-

lates a transition matrix probability during the explo-

ration of the environment in one sense, by shrinking

the alternatives for the robot. A similar procedure will

be investigated to renormalize and extend the transi-

tion probability matrix.

We also plan to extend our algorithms for scenar-

ios in which heterogeneousrobots select tasks accord-

ing to their skills, that represent different sensing ca-

pabilities for identifying objects of interest. Finally,

we are working on the implementation of MRTA al-

gorithms for a team of physical robots for landmine-

like object detection.

REFERENCES

Botelho, S. and Alami, R. (1999). M+: A scheme for multi-

robot coopeation through negotiated task allocation

and achievement. In Proc. Int. Conf. on Robotics and

Automation, pages 1234–1239, Detroit, MI.

Bullo, F., Frazzoli, E., Pavone, M., Savla, K., and Smith, S.

(2011). Dynamic vehicle routing for robotic systems.

In Proc. of the IEEE, volume 99, pages 1482–1504.

Dahl, T., Matari´c, M., and Sukhatme, G. (2009). Multi-

robot task allocation algorithm based on vacancy

chain scheduling. Robotics and Autonomous Systems,

57(6-7):674–687.

Dasgupta, P. (2011). Multi-robot task allocation for per-

forming cooperative foraging tasks in an initially un-

known environment. In Jain, L., editor, Innovations

in Defense Support Systems - 2, pages 5–20. Springer-

Verlag.

Dias, M. (2004). TraderBots: A New Paradigm for Ro-

bust and Efﬁcient Multirobot Coordination in Dynam-

ics Environments. PhD thesis, The Robotics Institute,

Carnegie Mellon University.

Dias, M. B., Zlot, R., Kalra, N., and Stentz, A. (2006).

Market-based multirobot coordination: a survey and

analysis. In Proceedings of the IEEE, Special Issue on

Multirobot Systems, volume 94, pages 1257– 1270.

Gerkey, B. and Matari´c, M. (2004). A formal analysis

and taxonomy of task-allocation in multi-robot sys-

tems. The International Journal of Robotics Research,

23(9):939–954.

Huang, X. and Serfozo, R. (1999). Spatial queueing

processes. Mathematics of Operations Research,

24(4):865–886.

Jones, E., Browning, B., Dias, M., Argall, B., Veloso, M.,

and Stentz, A. (2006). Dynamically formed hetero-

geneous robot teams performing tightly coordinated

tasks. Proceedings of the International Conference on

Robotics and Automation.

Jones, E., Dias, M., and Stentz, A. (2011). Time-extended

multi-robot coordination for domains with intra-path

constraints. Autonomous robots, 30(1):41–56.

Lagoudakis, M., Markakis, V., Kempe, D., Keskinocak,

P., Koenig, S., Kleywegt, A., and Tovey, C. (2005).

Auction-based multi-robot routing. In Proc. of the In-

ternational Conference on Robotics: Science and Sys-

tems, pages 343–350, Boston, MA.

Liu, L. and Shell, D. (2011). Assessing optimal as-

signment under uncertainty: An interval-based algo-

rithm. The International Journal of Robotics Re-

search, 30(7):936–953.

Mataric, M., Sukhatme, G., and Ostergaard, E. (2003).

Multi-robot task allocationin uncertain environments.

Autonomous Robots, 14:255–263.

Miller, D., Dasgupta, P., and Judkins, T. (2006). Dis-

tributed task selection in multi-agent based swarms

using heuristic strategies. In SAB’06 Proceedings of

the 2nd international conference on Swarm robotics,

pages 158–172. Springer-Verlag.

Schneider, J., Apefelbaum, D., Bagnell, D., and Simmons,

R. (2005). Learning opportunity costs in multi-robot

market based planners. In Proceedings of the Interna-

tional Conference on Robotics and Automation, pages

1151 – 1156, Barcelona, Spain.

Tang, F. and Parker, L. (2007). A complete methodology for

generating multi-robot task solutions using asymtre-

d and market-based task allocation. In Proceedings

of the International Conference on Robotics and Au-

tomation, pages 3351–3358, Rome, Italy.

Viguria, A., Maza, I., and Ollero, A. (2007). Set: An al-

gorithm for distributed multirobot task allocation with

dynamic negotiation based on task subsets. In Pro-

ceedings of the International Conference on Robotics

and Automation, pages 3339–334, Rome, Italy.

Zlot, R. (2006). Complex Task Allocation for Multi-robot

Teams. PhD thesis, Carnegie Mellon University.

AStochasticQueueingModelforMulti-robotTaskAllocation

261