Physiological Data Validation of the Hexoskin Smart Textile
N. H. Cherif
1,3
, N. Mezghani
2,3
, N. Gaudreault
4
, Y. Ouakrim
2,3
, I. Mouzoune
3,4
and P. Boulay
5
1
Faculty of Science and Technology, University of Lorraine, France
2
LICEF Research Center, TELUQ University, Montréal, Quebec, Canada
3
Laboratoire de Recherche en Imagerie et Orthopédie (LIO), Centre de Recherche du CHUM, Montréal, Quebec, Canada
4
Faculté de Médecine et des Sciences de la Santé, Université de Sherbrooke, Quebec, Canada
5
Faculté des Sciences de L’activité Physique, Université de Sherbrooke, Quebec, Canada
Keywords:
Concordance Correlation Coefficient, Intraclass Correlation Coefficient, Bland-Altman, Agreement Analysis,
Cardiorespiratory.
Abstract:
The aim of this study is to validate cardiorespiratory function measurement of a healthy population provided
by a wearable textile during a progressive maximal exercise test. The following measurements were collected
using embedded sensors to assess three variables: heart rate (HR), breathing rate (BR) and ventilation (Ve).
These variables were recorded simultaneously by the wearable textile and using as a reference system for
a comparison purpose. The validation was performed based on the two systems agreement estimation by
calculating the intraclass correlation coefficient (ICC), the concordance correlation coefficient (CCC) and the
Bland-Altman plot for each variable. Twenty-eight healthy volunteers participated in this study. Analysis
of each participant under exercise condition by the two measurement systems revealed high CCC values (ρ
c
between 0.91 and 0.99), no deviation from the 45
line (C
b
between 0.96 and 0.99) and significant ICC values
(ρ between 0.91 and 0.99, p < 0.05) for HR and BR. The Bland Altman plot for HR and BR indicated no
deviation of the mean difference from zero and a small variability with tight agreement limits. However, the
analysis of the estimated ventilation Ve of each participant revealed doubtful values for the CCC (ρ
c
between
0.2 and 0.99) and ICC (ρ between 0.11 and 0.99). In summary, the Hexoskin presented good agreement for
HR and BR. However, for ventilation, it is difficult to conclude from the results due to variability.
1 INTRODUCTION
Technological advances have offered new solutions to
monitor and record physiological information such as
heart rate, breathing rate and physical activity levels,
from sensors embedded in wearable clothing.These
technological solutions are growing and evolving
rapidly to become new trends (Paradiso et al., 2005).
Hexoskin
1
(Carre Technlogies Inc, Quebec,
Canada) is an intelligent shirt with flexible sensors
sewn directly into the fabric and it is perfectly ma-
chine washable. It is embedded with several sensors:
(1) Activity sensors (Figure 1 (a)), which record the
movements (in three dimensions) and the acceleration
of the user, (2) Respiration sensors (Figure 1 (b)), in
the form of two rings present in the chest and abdom-
inal. By analyzing the user’s thoracic and abdominal
breathing data, Hexoskin can calculate the number of
breaths per minute and the volume of air consumed
1
http://www.hexoskin.com/
(in L/min), and (3) Heart sensors (Figure 1 (c)), in the
form of electrodes, can act as an electrocardiogram
ECG and deliver the heart rhythm of the intelligent
textile wearer (in number of beats per minute) (Fig-
ure 1). The intelligent textile is able to measure levels
of physical activity and physiological signals in real
time. Using Bluetooth technology, these signals can
be monitored remotely from a smart device.
The Hexoskin has been developed initially for
high-level sports. However, there is a current grow-
ing interest for its use in clinical settings. Hence, the
physiological data provided by the Hexoskin need to
be validated. A few studies have considered such a
validation. To validate the Hexoskin wearable shirt
during lying, sitting, standing, and walking activi-
ties, a sample of 20 participants have been consid-
ered (Villar et al., 2015). The study revealed a low
coefficient of variation and high intraclass correlation
values for heart rate, breathing rate, and hip motion
intensity while comparing the Hexoskin and labora-
tory standard devices. The magnitude of changes in
150
Cherif, N., Mezghani, N., Gaudreault, N., Ouakrim, Y., Mouzoune, I. and Boulay, P.
Physiological Data Validation of the Hexoskin Smart Textile.
DOI: 10.5220/0006588001500156
In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 1: BIODEVICES, pages 150-156
ISBN: 978-989-758-277-6
Copyright © 2018 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
(a)
(b)
(c)
Figure 1: The Hexoskin acquisition system: (a) Activity
sensor, (b) Respiration sensor and (c) Heart sensor.
tidal volume and minute ventilation from the Hex-
oskin closely tracked those measured by the labora-
tory standard method. In another study, to measure
cardiorespiratory fitness, the Hexoskin has been used
in a sample of 10 participants using a hiking situa-
tion during two field activities at different intensities
(Montes et al., 2015). No significant differences for
trail types were noted for maximal heart rate and res-
piratory rate. This is due to many missed data dur-
ing the data collection. In a recent study, to moni-
tor heart rate, twelve male volunteers wore the Hex-
oskin and Polar H7 heart rate sensor and they com-
pleted variable physical activities, using a stationary
training bicycle under two different climate condi-
tions (Al Sayed et al., 2017). The study revealed a
high correlation and an absence of significant differ-
ences between the two systems in monitoring the sub-
jects’ heart rates. In another study, to investigate the
validity and reliability of the Hexoskin vest for mea-
suring respiration and heart rate, ten male elite cy-
clists conducted a maximal aerobic cycle ergometer
test using a ramped protocol (Elliot et al., 2017). Au-
thors conclude that the Hexoskin vest is sufficiently
valid and reliable for measurements of breathing fre-
quency and heart rate in elite athletes but the calcu-
lated minute ventilation value produces during such
exercise should be used with caution due to the lower
validity and reliability of this variable. Finally, the
maximum oxygen uptake (VO
2max
) has been evalu-
ated by (Bassett Jr and Howley, 2000; Hawkins et al.,
2007). This parameter, which can be defined as the
maximum amount of oxygen that the body consumes
during an intense effort (Howley et al., 1995), is con-
sidered as the best indicator of cardiorespiratory func-
tion. Moreover, an important parameter in determin-
ing the quality of a medical instrument is the agree-
ment with a gold standard. These previous studies
have been performed on a small sample size which
can seriously limit the interpretation of the results and
the validation process. In addition, in some cases
(Al Sayed et al., 2017; Elliot et al., 2017), the data col-
lection has been performed on stationary bikes which
does not take into account the usual movements of the
upper body.
In this study, we compared cardiorespiratory pa-
rameters from the Hexoskin against standard labo-
ratory devices during a progressive maximal exer-
cise test. Therefore, the purpose was to validate car-
diorespiratory parameters measured simultaneously
by Hexoskin and a laboratory standard devices dur-
ing a progressive maximal exercise test according
to BSU/Bruce ramp (Kaminsky and Whaley, 1998)
protocol. More precisely, we will compare heart
rate (HR) from the Hexoskin and heart rate from a
reference laboratory electrocardiogram (ECG). The
breathing rate (BR) and the estimated ventilation (Ve)
will be compared between Hexoskin and a reference
laboratory metabolic cart.
2 METHODOLOGY
2.1 Participants
Thirty healthy volunteers (18 males and 12 females)
participated in the study. For 2 participants, the mea-
surements were removed from the database due to
technical issues that occurred when recording of the
sensors signals that were not in the correct format.
The problem was reported to Hexoskin for investi-
gation. This gave a total of twenty-eight volunteers
(17 males and 11 females, age 27.07± 7.25 years old,
weight 73.73 ± 13.41 kg, heigh 173.12 ± 8.15 cm,
Body Mass Index BMI 24.5 ± 3.44).
The study was approved by institutional ethics
committees, with all subjects providing written in-
formed consent before their participation. All partic-
ipants completed a sociodemographic questionnaire
and the Q-AAP + questionnaire to confirm that they
were able to perform a vigorous exercise.
2.2 Data Collection
Data collection was performed at the University of
Sherbrooke exercise physiology laboratory during a
progressive maximal exercise test. Each participant
wore the smart textile Hexoskin while they were sub-
mitted to a progressive maximal exercise test using
the Cosmed Quark metabolic cart and 12 lead elec-
trocardiogram which are considered gold standard in
an exercise physiology lab.
The test was performed according to the
BSU/Bruce ramp protocol (Kaminsky and Whaley,
1998), a testing protocol commonly used in clinical
Physiological Data Validation of the Hexoskin Smart Textile
151
and research settings. The treadmill grade starts at
0% and the walking pace at 1.7 mph. Every 20 sec-
onds, the treadmill speed increases gradually (by 0 to
0.1 mph) and/or grade increases gradually (by 0 to
0.5%). Every 3 minutes during this ramp protocol,
the work rates were identical to those of the standard
Bruce protocol. Ratings of perceived exertion was as-
sessed throughout the maximum cardiorespiratory ef-
fort test using the Borg scale of 6-20 (Borg, 1982).
The recorded data (HR, BR and Ve) of the two
devices were averaged every 20 seconds leading to a
time series data of each variable. Figure 2 illustrates,
for one participant the measurements of the heart rate
(Figure 2 (a)), breathing rate (Figure 2 (b)) and venti-
lation (Figure 2 (c)) using the hexoskin and using the
reference equipment Cosmed.
0
50
100
150
200
20 60 100 140 180 220 260 300 340 380 420 460 500 540 580 620 660 700 740 780
(a)- Heart-rate- (bpm)
Cosmed
Hexoskin
Time%(seconds)
0
10
20
30
40
50
60
20 60 100 140 180 220 260 300 340 380 420 460 500 540 580 620 660 700 740 780
(b)-Breathi ng- rate-(/min)-
Cosmed
Hexoskin
Time%(seconds)
0
20
40
60
80
100
120
140
160
20 60 100 140 180 220 260 300 340 380 420 460 500 540 580 620 660 700 740 780
(c)-Ventilation-(L/min)
Cosmed
Time%(seconds)
Figure 2: (a) Heart rate, (b) breathing rate and (c) ventila-
tion measurements using the Hexoskin (red color) and using
the reference equipment Cosmed (blue color) for one par-
ticipant. The x-axis represents the averaged values for each
variable every 20 seconds.
A preliminary analysis of the collected data shows
the presence of outlier data for one participant (par-
ticipant 12). Indeed the participant is muscular and
when he was to run towards the end of the test,
the pectorals make complex movements (bouncing)
and the Hexoskin’s cardiac sensors detached from the
body which leading to abnormal recording (as illus-
trated in Figure 3. In consequence, the participant
is considered as an outlier and was deleted from our
database.
0
20
40
60
80
100
120
140
160
180
200
Heart&rate&(Participant&12)&
Cosm ed
Hexoskin
Time&(seconds)
Figure 3: Heart rate curves for the Hexoskin and the labo-
ratory reference equipment for participant 12.
2.3 Statistical Analysis for Physiological
Data Validation
An important parameter in determining the quality
of the wearable textile Hexoskin is the agreement
with a gold standard. Agreement means the accu-
racy of this new measurement method (de Vet H.C.W.,
1998). Several statistical methods were used to test
the agreement of medical devices with quantitative re-
sults (Bland and Altman, 1983; Luiz RR, 2005).
In this study, the agreement (concordance) be-
tween the two methods of measuring Heart Rate
(HR), Breathing Rate (BR) and Ventilation (Ve) were
estimated by calculating the Intraclass Correlation
Coefficient (ICC) and the Concordance Correlation
Coefficient (CCC). The 95% confidence interval was
estimated for ICC.
We also used the Bland Altman method to nuance,
invalidate, or confirm the level of concordance quan-
tified by numerical methods. Bland-Altman (Bland
and Altman, 1986) is a method used to compare two
measurements of the same variable. It is based on the
quantification of the agreement between two quanti-
tative measurements by studying the mean difference
and constructing limits of agreement.
2.3.1 Intra-class Correlation Coefficient (ICC)
The intra-class correlation coefficient (ICC) measures
the agreement and assess the reliability of medical in-
strument continuous outcomes. It was originally pro-
posed by Fisher (Fisher, 1925), who suggested to use
an analysis of variance with a separation of within-
subject and between-subject variability. There are dif-
ferent forms of ICCs. Shrout and Fleiss have pre-
BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices
152
sented six forms of ICCs (Shrout and Fleiss, 1979)
and McGraw and Wong have presented ten forms
(McGraw and Wong, 1996). The ICC is defined as
the following ratio,
ρ =
σ
2
b
σ
2
b
+ σ
2
w
(1)
where, σ
2
b
denotes the variance between subjects, and
σ
2
w
denotes the variance within subjects. The lower
the within subjects variability compared to the vari-
ability between subjects, the greater the intra-class
correlation coefficient. The ICC varies between 0 and
1. When reliability is high, ICC is close to 1 and when
reliability is low, ICC is close to 0. It’s recommended
to calculate the confidence intervals (CI) for the ICC
(Shrout and Fleiss, 1979). In addition, p-values of
ICC can be used to evaluate the accuracy and signifi-
cance of the ICC. We consider as reliable an ICC with
a p-value 0.05.
2.3.2 Concordance Correlation Coefficient
(CCC)
The concordance correlation coefficient (CCC) quan-
tifies the agreement between a new measurement
method and a gold standard measurement method
(Lin L, 1989; Lin and Kuei, 2000). The CCC com-
bines measures of both precision and accuracy. Pre-
cision is measured with the correlation coefficient r.
Model accuracy is evaluated with the bias correction
factor C
b
. A C
b
equal to 1 indicates no deviation
from the 45
line. The multiplication of both values
gives both precision and accuracy at the same time.
To be perfectly concordant, two measurement tech-
niques have to be perfectly correlated and also pro-
vided identical measurements, i.e., situated on the 45
line through the origin.
Let x
1
= (x
1
, x
2
, ..., x
n
) denote the series of mea-
surement of the Hexoskin and y
2
= (y
1
, y
2
, ..., y
n
) de-
note the series of measurement of the reference labo-
ratory equipment for a specific participant. The CCC
is defined as,
ρ
c
=
2σ
12
σ
2
1
+ σ
2
2
+ (µ
1
µ
2
)
2
= rC
b
(2)
where µ
1
and µ
2
denote the means of each series
of measurements, σ
1
and σ
2
are the corresponding
variances, and σ
12
is the covariance between the two
series of measurements. The CCC takes values be-
tween -1 and 1, where these values mean respectively
a perfect discordance and a perfect concordance.
McBride (McBride, 2005) suggests the following
descriptive scale for values of the concordance
correlation coefficient:
Value of ρ
c
Strength of agreement
>0.99 Almost perfect
0.95 - 0.99 Substantial
0.90 - 0.95 Moderate
<0.90 Poor
3 RESULTS AND DISCUSSION
Experimental results were obtained using data from
28 healthy participants (as described in section 2.1).
The variables of interest are the heart rate (HR),
breathing rate (BR) and estimated ventilation (Ve),
which were measured simultaneously by Hexoskin
and a laboratory reference equipment. CCC and ICC
were calculated for the three variables of interest HR,
BR and Ve for each participant. Simulations were per-
formed using packages of the open source program-
ming language and data science environment R.
Figures 4, 5 and 6 show the frequency distribu-
tion of the obtained intra-class correlation coefficient
(ICC) and concordance correlation coefficient (CCC)
values from all participants.
The HR measured simultaneously for each par-
ticipant by the Hexoskin and the reference labora-
tory equipment revealed high CCC values (ρ
c
be-
tween 0.98 and 0.99), and significant ICC values (ρ
between 0.98 and 0.99, p < 0.05) for almost all par-
ticipants (Figure 4). All ICC values are in the 95%
confidence interval except for one participant (partic-
ipant 12) who has a low CCC (ρ
c
= 0.58) and a low
ICC (ρ = 0.59, p-value < 0.05).
Figure 4: Histogram of the CCC and ICC values for heart
rate, for all participants.
Physiological Data Validation of the Hexoskin Smart Textile
153
Figure 5: Histogram of the CCC and ICC values for breath-
ing rate, for all participants.
Figure 6: Histogram of the CCC and ICC values for Venti-
lation, for all participants.
The analysis of BR of each participant under ex-
ercise condition by the two measurement systems re-
vealed high CCC values (ρ
c
between 0.91 and 0.99)
and significant ICC values (ρ between 0.91 and 0.99,
p < 0.05) (Figure 5). All ICC values are in the 95%
confidence interval.
The estimated ventilation (Ve) measured simulta-
neously by the Hexoskin and the laboratory reference
equipment during a progressive maximal exercise test
revealed doubtful values for each of the CCC (ρ
c
be-
tween 0.2 and 0.99), ICC (ρ between 0.11 and 0.99)
and C
b
between 0.2 and 0.99 (Figure 6). It is noted
that there is a significant difference between the two
groups male and female in relationship to the CCC
and ICC values. We observed that 8 out of 11 women
(73%) had CCC values greater than 0.9, this goes to
moderate to an almost perfect agreement according to
the adopted scale (McBride, 2005). This can be ex-
plained by the fact that the two intelligent textiles are
not made the same way for men and women.
Figure 7 shows the deviation from the 45
line.
Recall that the model accuracy is evaluated with the
bias correction factor C
b
. A C
b
equal to 1 indicates
no deviation from the 45
line. Figure 7 (example
1) shows no deviation from the 45
line (C
b
more
than 0.99) for heart rate for one of the participants
(all participants have almost the same graphs). With
an average of ρ
c
of more than 0.99, this corresponds
to an almost perfect agreement with the scale pro-
posed by McBride (2005). The results obtained with
the numerical methods were confirmed by the Bland-
Altman method. The Bland Altman results indicated
no deviation of the bias (mean difference) from zero
and relatively small variability with tight agreement
limits.
Figure 7 (example 2) shows no deviation from the
45
line (C
b
more than 0.99) for breathing rate for
one of the participants (all participants have almost
the same graphs). With an average of ρ
c
of more
than 0.97, this corresponds to a substantial agree-
ment. The results obtained with the numerical meth-
ods were confirmed by the Bland-Altman method, the
bias (mean difference) is close to 0 for all participants
and the agreement limits are fairly tight. Figure 7 (ex-
ample 3) shows no deviation from the 45
line for
ventilation for one of the participants (female). How-
ever, example 4 (Figure 7) shows a large deviation of
the 45
line for one of the participants (male).
4 CONCLUSIONS
In this study, we tested the sensor data extracted from
the wearable textile Hexoskin. Specifically, we val-
idated the cardiorespiratory variables measured with
Hexoskin and with laboratory standard metabolic cart
during a progressive maximal exercise test.
Using the Intraclass Correlation Coefficient, the
Concordance Correlation Coefficient and the Bland
Altman, we calculate the agreement between the two
methods which revealed good agreement and low
variability for heart rate and breathing rate. Hexoskin
provides ventilation values from estimates, which is
an important limitation of the Hexoskin.
The heart rate values are more accurate because
they are calculated directly from the ECG signal using
BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices
154
Example 1: Heart rate beats per min
Example 3: Ventilation in L/min (Female) Example 4: Ventilation in L/min (Male)
Cosmed
Cosmed
CosmedCosmed
Hexoskin
Hexoskin
Hexoskin
Hexoskin
Example 2: Breathing rate per min
-- 45° line!
--- Regression line!
Figure 7: Graphical representation of the results of the two measurement systems for heart rate (example 1), breathing rate
(example 2) and ventilation for one of the women and one of the men (examples 3 and 4). The 45
line represents the perfect
match with the reference method.
the integrated electrodes. Same for the breathing rate
which is calculated directly from the two integrated
respiration sensors in the thoracic and abdominal lev-
els. However, ventilation is an estimation provided
using the two integrated respiration sensors.
This study confirm the good agreement for heart
rate and breathing rate. However, for the ventilation,
it remains difficult to conclude on the validity of Hex-
oskin because the results are variable.
ACKNOWLEDGEMENTS
The authors would like to thank the Natural Sciences
and Engineering Research Council (NSERC), and the
Canada Research Chair.
REFERENCES
Al Sayed, C., Vinches, L., and Hallé, S. (2017). Validation
of a wearable biometric systemâ
˘
A
´
Zs ability to moni-
tor heart rate in two different climate conditions under
Physiological Data Validation of the Hexoskin Smart Textile
155
variable physical activities. E-Health Telecommunica-
tion Systems and Networks, 6(02):19.
Bassett Jr, D. R. and Howley, E. T. (2000). Limiting factors
for maximum oxygen uptake and determinants of en-
durance performance. Medicine & Science in Sports
& Exercise, 32(1):70.
Bland, J. M. and Altman, D. (1983). Measurement in
medicine: the analysis of method comparison studies.
The Statistician, 32:307–317.
Bland, J. M. and Altman, D. (1986). Statistical methods for
assessing agreement between two methods of clinical
measurement. The lancet, 327(8476):307–310.
Borg, G. (1982). Psychophysical bases of perceived ex-
ertion. Medicine & Science in Sports & Exercise,
14(5):377–381.
de Vet H.C.W. (1998). Observer reliability and agreement.
John Wiley & Sons Ltd, pages 3123–3128.
Elliot, C. A., Hamlin, M. J., and Lizamore, C. A. (2017).
Validity and reliability of the hexoskin (r) wearable
biometric vest during maximal aerobic power testing
in elite cyclists. The Journal of Strength & Condition-
ing Research.
Fisher, R. A. (1925). Theory of statistical estimation.
Proceedings of the Cambridge Philosophical Society,
22(5):700–725.
Hawkins, M. N., Raven, P. B., Snell, P. G., Stray-
Gundersen, J., and Levine, B. D. (2007). Maximal
oxygen uptake as a parametric measure of cardiores-
piratory capacity. Medicine & Science in Sports &
Exercise, 39(1):103–107.
Howley, E. T., Bassett, D. R., and Welch, H. G. (1995).
Criteria for maximal oxygen uptake: review and com-
mentary. Medicine and science in sports and exercise,
27(9):1292–1301.
Kaminsky, L. A. and Whaley, M. H. (1998). Evaluation of a
new standardized ramp protocol: the bsu/bruce ramp
protocol. Journal of Cardiopulmonary Rehabilitation
and Prevention, 18(6):438–444.
Lin, L. and Kuei, I. (2000). A note on the concordance
correlation coefficient. Biometrics, 56:324–325.
Lin L, K. I. (1989). A concordance correlation coefficient to
evaluate reproducibility. Biometrics, pages 255–268.
Luiz RR, S. M. (2005). More than one statistical strategy to
assess agreement of quantitative measurements may
usefully be reported. Journal of Clinical Epidemiol-
ogy, 58(3):215–216.
McBride, G. (2005). A proposal for strength-of-agreement
criteria for lin’s concordance correlation coefficient.
NIWA Client Report: HAM2005-062.
McGraw, K. O. and Wong, S. P. (1996). Forming inferences
about some intraclass correlation coefficients. Psycho-
logical methods, 1(1):30.
Montes, J., Stone, T. M., Manning, J. W., McCune, D.,
Tacad, D. K., Young, J. C., DeBeliso, M., and Navalta,
J. W. (2015). Using hexoskin wearable technology to
obtain body metrics during trail hiking. International
journal of exercise science, 8(4):425.
Paradiso, R., Loriga, G., and Taccini, N. (2005). A wearable
health care system based on knitted integrated sen-
sors. IEEE transactions on Information Technology
in biomedicine, 9(3):337–344.
Shrout, P. E. and Fleiss, J. L. (1979). Intraclass correla-
tions: uses in assessing rater reliability. Psychological
bulletin, 86(2):420.
Villar, R., Beltrame, T., and Hughson, R. L. (2015). Vali-
dation of the hexoskin wearable vest during lying, sit-
ting, standing, and walking activities. Applied Physi-
ology, Nutrition, and Metabolism, 40(10):1019–1024.
BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices
156