TOOTH-TOUCH SOUND AND EXPIRATION SIGNAL

DETECTION AND ITS APPLICATION IN A MOUSE

INTERFACE DEVICE FOR DISABLED PERSONS

Realization of a Mouse Interface Device Driven by Biomedical Signals

Koichi Kuzume

Information Science and Technology Department, Yuge National College of Maritime Technology

1000 Shimoyuge, Kamishima-cho, Ochi-gun Ehime-ken, Japan

Keywords: Disabled persons, Tooth-touch sound, Expiration signal, Signal detection, Mouse.

Abstract: Presented is a mouse interface device for disabled persons using tooth-touch sound and expiration signals. It

enables disabled persons to operate a personal computer easily using a mouse driven by their tooth-touch

and expiration. A bone conduction microphone was used to detect the tooth-touch sound and the piezo film

sensors to sense the expiration. Both sensors had superior features including being easy to handle, light

weight, user-friendly, and inexpensive making the device practical as a mouse interface for disabled

persons. First, we describe the novel method for detecting the tooth-touch sound in conjunction with Dyadic

Wavelet Transform to improve the performance of tooth-touch sound detection. The device consists of

sensor units that can sense the tooth-touch sound and the expiration signals, an individual adaptive circuit,

and an output interface to connect directly with a mouse and Environmental Control System (ECS). Next,

we designed the device using Hardware Description Language (VHDL) and realized a prototype of mouse

interface with a Field Programmable Gate Array (FPGA) in practice. Finally, we confirmed the basic

operation of the mouse.

1 INTRODUCTION

Thanks to advances in electronics and information

technology, everybody can use a personal computer

easily. This includes disabled persons who are

interested in using the Internet and multimedia. A

mouse is usually used as a computer interface to

select the icons on a display and execute program by

clicking the mouse button. As a result, it has become

essential for disabled persons to use a mouse.

However, as hand operation is needed to control a

mouse, disabled persons may not be able to use a

mouse easily, making development of alternative

input devices necessary.

Several types of mouse for disabled persons have

been devised. For example, Dimitry et. al. developed

a mouse device using vision-based technology

(Dimitry, 2004). The mouse cursor position could be

controlled by multiple eye blinks and nose

movement used for clicking operations. However, it

was sensitive to external disturbance such as the

brightness of the room and users' movement.

Recently, we proposed a hands-free man-

machine interface device, utilizing the expiration in

conjunction with the tooth-touch sound signal,

which we have been researching for realization (K.

Kuzume, 2006, 2008, 2010). This device utilized the

bone conduction signal collected by a bone-

conduction microphone, which is utilized for

clicking operation of the mouse and the expiration

signal gathered by piezo-film sensors to control the

mouse cursor position. The proposed device met

conditions required by disabled persons namely low

price, fitness, and ease of handling. We constructed

a prototype device and tested its usefulness as an

input device for a character input system (K.

Kuzume, 2008). However, there are some remained

problems with issues, such as the reliability of the

input operation by the tooth-touch sound signal and

the realization of the other applications, such as in

an ECS with better usability by means of our input

device. In practical application, since the amplitude

of the tooth-touch sound signal varies between

people, detecting even small signals as accurately as

possible is required along with elimination of the

voice signal. The input operation for the disabled

Kuzume K..

TOOTH-TOUCH SOUND AND EXPIRATION SIGNAL DETECTION AND ITS APPLICATION IN A MOUSE INTERFACE DEVICE FOR DISABLED

PERSONS - Realization of a Mouse Interface Device Driven by Biomedical Signals.

DOI: 10.5220/0003357300150021

In Proceedings of the 1st International Conference on Pervasive and Embedded Computing and Communication Systems (PECCS-2011), pages 15-21

ISBN: 978-989-8425-48-5

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

persons to control the ECS smoothly is desired to be

as intuitive as possible.

In this paper, we propose a novel method for

eliminating the voice and white noise suppression by

dyadic wavelet transform in conjunction with the

signal adaptive threshold technique and show that

our method has excellent performance. Next, to

improve the usability of the input device, we

modified the control method to adjust the mouse

cursor position more intuitively, adapting to the

amplitude of the expiration signal. Finally, we

designed the tooth-touch sound and the expiration-

based mouse device system using VHDL and

realized the system on an FPGA chip in practice.

This paper is organized as follows: In section 2,

we detailed the novel method for detecting the tooth-

touch sound using a dyadic wavelet based noise

suppression method and review the expiration signal

detection method briefly. In Section 3 we present the

device architecture of a mouse driven by the tooth-

touch sound and expiration signals. We are then

devoted to the design of the mouse interface device

and realization of it by an FPGA chip. In Section 4,

we apply our device to control of mouse cursor

position by expiration signal and confirmed its basic

operation. Section 5 outlines our conclusions and

potential development.

2 SIGNAL DETECTION

2.1 Review of Tooth Touch Sound

Signal Detection Technique

Several kinds of noises, such as voice and ham

noise, interfere with tooth-touch sound detection, the

most serious being voice noise and white noise. The

bone conduction microphone picked up not only the

tooth-touch sound, but also the user’s voice.

Development of the voice elimination method is

required to eliminate faults originating from

background noise.

Our analysis on the tooth-touch sound signal has

shown that the frequency spectrum of tooth-touch

sound is overlapped with that of voice signal.

Therefore it is difficult to detect only the tooth-touch

sound in the measured signal by the conventional

band pass filters. Moreover, the magnitude of the

tooth-touch sound signal varies between people. If

the amplitude of the tooth-touch sound signal is too

small, it is necessary to amplify the signal. As results,

the tooth-touch sound signal may be corrupted by

white noise.

In this section, we propose the novel method for

eliminating voice signal, which is very simple and

easy to realize by simple circuit. Moreover, we also

present dyadic wavelet transform for the white noise

suppression.

-500

-400

-300

-200

-100

100

200

300

400

0 5000 10000 15000 20000 25000 30000 35000

oice

Figure 1: The bone-conduction signal containing voice,

tooth-touch sound, and white noise.

2.1.1 Voice Elimination Method

Figure 1 shows the signal containing both the voice

and tooth-touch sound. The tooth-touch sound and

voice were rarely generated at the same time. The

tooth touch sound clearly resembled an impulse

signal, having higher frequency components

comparing with the voice signal and a distinct

pattern. The voice eliminating method involved

calculating the average of the absolute value of the

signal.

We depict the distribution of the amplitude of

the voice signal in Figure 2(a) and also distribution

of its absolute value in Figure 2(b). According the

previous researches on the voice signal, it is known

that its distribution function follows to normal

distribution. The statistics theory tells that in the

normal distribution function, sampled data without

8±x

occupies less than

[%]107.6

14−

, where

average value of the voice signal and σis standard

deviation of the distribution shown in Figure 2(a).

Now we set a threshold level adapting to the signal

for eliminating the voice signal. If the threshold

level set to 8σ, most of the voice signal can be

eliminated. However, to compute the standard

deviation, multipliers circuits and a large amount of

computation are needed. We surveyed the

relationship between the standard deviation σ and

the average

of the absolute value of the voice

signal. We then confirmed there is nearly linear

relationship between them in (1).

μσ

5.1≅

(1)

By using (1), we can easily calculate the standard

deviation by the average value of the absolute value.

In the actual system we choose sampling frequency

Tooth-touch sound signals

PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems

at 4 KHz, and average interval time T in 0.128sec

(512 samples), then

is calculated by (2),

512

511

∑

(2)

where

denotes the bone-conduction signal at the

time i .

Now, if we set the threshold level to 8σ, we can get

the following equation (3).

μμσ

125.188 =×≅=

(3)

Considering the margin and simple calculation, we

set V

to 16

, finally signal adaptive threshold

level can be given by (4).

−

×=

∑

−

ith

(4)

The threshold level can be obtained by calculating

sum of 2

samples and then shifting the sum by 5-bit

toward to the right. Figure 3 (a), (b) show the

threshold level and the result of the tooth-touch

sound signal detection by computer simulations. The

simulation results show that the only the tooth-touch

sound signals can be detected correctly.

100

150

200

250

300

350

400

450

500

-100 -80 -60 -40 -20 0 20 40 60 80 100

(a)

100

200

300

400

500

600

700

800

900

1000

0 102030405060708090100

(b)

Figure 2: (a) Distribution of the amplitude of the voice

signal. (b) Distribution of the absolute value of Figure 2

(a).

Figure 3: (a) Signal adaptive threshold (b) Result of tooth-

touch sound signal detection.

2.1.2 White Noise Suppression by the DWT

The magnitude of the tooth-touch sound signal

varied with individual characteristics, including age,

sex, and continuous operating time. It was necessary

for the system to detect even small tooth-touch

sounds as accurately as possible. When the

amplitude of the tooth-touch sound matched that of

the white noise, the bone conduction signal needed

to be amplified. White noise made it difficult to

detect the tooth-touch sounds signal correctly using

the earlier method in 2.1.1. Figure 4 shows a block

diagram of the DWT, threshold processing, and

inverse dyadic wavelet transform. It was desirable

for filter length to be as short as possible to reduce

computation time. We employed the Haar wavelet

filter, with a filter length of 2.The threshold level

can be set shown in (5), where

is deviation of

the white noise and N is the number of data

(D.L.Donoho,1995). The lowpass decomposition

signals within

V± can be regarded as the white

noise and be set to zero. As the results of this signal

processing the white noise could be suppressed and

tooth-touch sound signal is remained.

eTH

log2

(5)

TOOTH-TOUCH SOUND AND EXPIRATION SIGNAL DETECTION AND ITS APPLICATION IN A MOUSE

INTERFACE DEVICE FOR DISABLED PERSONS - Realization of a Mouse Interface Device Driven by Biomedical

Signals

yadic wavelet transfor

Thr e sho l d

’

Dyadic inverse wavelet transform

yadic wavelet transfor

Thr e sho l d

’

yadic wavelet transfor

Thr e sho l d

’

Thr e sho l d

’

Dyadic inverse wavelet transform

a : Bone conduction signal (original signal)

′

a : Denoised signal,

: Highpass decomposition filter,

: Highpass reconstruction filter,

h : Lowpass decomposition filter,

: Lowpass reconstruction filer.

Figure 4: White noise suppression by the dyadic wavelet

transform.

Figure 5 shows the FPGA-based tooth-touch

sound detector with the above-mentioned noise

suppression functions. The tooth-touch sound

detector contains the white noise suppression and

voice elimination unit, outputting pulse signal at the

time, when only the tooth-touch sound signals are

input as shown in Figure 6. And also, even the small

tooth-touch sounds can be detected.

2.1.3 Evaluation of Detection Performance

To evaluate the effectiveness of white noise

suppression by the DWT, we implemented practical

use test. We asked 4 persons (healthy, 20 year-old

males) to touch their teeth together 50 times. The

duration and strength were arbitrary. All tooth-touch

sound signals were stored by a data recorder. Using

the recorded data, we compared detection accuracy

between non-white noise suppression and white

noise suppression. Table 1 shows the experimental

results of performance of tooth-touch sound signal

detection based on the experimental results. The

results show that the number of positive faults

decreased and detection performance was improved.

(a)

(b)

Figure 6: (a) Result of the tooth-touch sound signal

detection from the bone-conduction signal corrupted by

the voice signal, (b) Tooth-touch sound signal detailing.

Table 1: Comparison of detection performance.

0% 0%0%　2%

Ｄ

2% 0%4%

Ｃ

2% 2%2%

Ｂ

Negative FaultPositive FaultNegative FaultPositive FaultUser

Error Detection Ratio

(in case of using DWT)

Error Detection Ratio

(in case of not using DWT)

2.2 Review of Expiration Signal

Detection by Piezo Film Sensors

The details of the expiration signal detection were

described in our previous paper (K. Kuzume, 2010).



Detectedtooth‐touch

soundsignal

+

Bone‐conductionsignal

Comparator

Denoising

(by DWT )

5-bit

right-shift register

Maximumvaluedetection

Comparator

Sumof512samples

Figure 5: Circuit diagram for detecting the tooth-touch sound signal.

PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems

We review it briefly in this section.

We set the piezo film sensors in the form of a

sensor array to detect the expiration signal. The film

had an area of about 13

×25mm

, suitable hardness

to sense the expiration and weighed less than 1g

(http//:www.t-sensor.co.jp). To detect the user’s

direction of expiration, three piezo film sensors were

set about 6cm from the user’s mouth, to the left,

center and right. Each sensor had two functions, one

being to detect the piezoelectric signal (vibration),

and the other to sense the pyroelectric signal

(temperature variation). A user fitted with 3-ch

piezo film sensors to detect the expiration signals

from three directions, right, center, and left.

Expiration could be accurately detected using both

the piezoelectric and pyroelectric signals to reduce

(a)

(b)

Figure 7: (a) Vibration component (piezoelectric) (b)

Temperature variation component (pyroelectric).

outside disturbance. This enabled us to dramatically

improve input efficiency by changing the direction

and duration of a user’s expiration. It was necessary

to accurately separate the piezoelectric and

pyroelectric signals from the original. Here we

present a novel method for signal separation. In

addition to the expiration signal, the signal obtained

by the sensors contained the DC offset and ham

noise. Only after eliminating these noises by dyadic

wavelet transform did we obtain the higher

frequency component of the expiration signal. Figure

7 shows the waveform of vibration component and

temperature variation component processed by the

DWT. These figures show that the pyroelectric

signal could be separated accurately from the

original signal.

Figure 8: Example of the expiration signal and Control

signals for moving the mouse cursor.

2.3 Control of Mouse Cursor Position

by Expiration Signal

We connected the proposed device to a personal

computer via a wheel-type mouse with a USB

interface. Two pairs of signals, moving the mouse

cursor in horizontal and vertical directions

respectively, were generated in the FPGA chip.

Figure 8 shows an example of the expiration signal

and a pair of signals for moving the mouse cursor.

The distance which a cursor is moved by the

expiration signal is dependent upon the strength and

duration of the expiration signal. When stronger

expiration is applied to the sensor, the more pulses

are output and as a result, the cursor moves quickly.

The phase deference between pair signals is 90

degrees.

3 DEVICE ARCHITECTURE

Architecture of our device is shown in Figure 9. The

device consists of a sensor unit, amplifiers, lowpass

filter, individual adaptive circuit, and output

interface for connecting with the ECS and a mouse

driver circuit. The sensor unit contains piezo film

sensors to detect the expiration and a bone

conduction microphone for detection of the

expiration and tooth-touch sound signals

respectively and ADC (Analog to Digital Converter).

The 3 piezo film sensors were set at the positions of

right, center, and left sides far from a user’s mouse

to detect not only the directions of a user’s

expiration but also the temperature variations by the

-0.3

-0.2

-0.1

0.1

0 5000 10000 15000 20000

[samples]

-3

-2

-1

0 5000 10000 15000 20000

[samples]

[V]

[au]

TOOTH-TOUCH SOUND AND EXPIRATION SIGNAL DETECTION AND ITS APPLICATION IN A MOUSE

INTERFACE DEVICE FOR DISABLED PERSONS - Realization of a Mouse Interface Device Driven by Biomedical

Signals

ＡＤ

変換器

呼気方向

呼気の強さ

呼気の長さ

の検出

歯音検出器

AMP/LPF

Bone-conduction Microphone

Piezo Film Sensor

FPGA

Rigｈｔ

Center

Left

ECS

熱変化検出（焦電効果）

個人特性記憶ユニット

Direction

Strength

Duration

Detection

Tooth-touch sound detection

Mouse control

Photo Coupler

AMP/LPF

FPGA

ECS

Temperature change

（Pyroelectric effect)

Memory for individual characteristics

AMP/LPF

ADC

Control Code Generator

ＡＤ

変換器

呼気方向

呼気の強さ

呼気の長さ

の検出

歯音検出器

AMP/LPF

Bone-conduction Microphone

Piezo Film Sensor

FPGA

Rigｈｔ

Center

Left

ECS

熱変化検出（焦電効果）

個人特性記憶ユニット

Direction

Strength

Duration

Detection

Tooth-touch sound detection

Mouse control

Photo Coupler

AMP/LPF

FPGA

ECS

Temperature change

（Pyroelectric effect)

Memory for individual characteristics

AMP/LPF

ADC

Control Code Generator

Figure 9: Schematic diagram of the tooth-touch sound and expiration based mouse interface.

pyroelectric effect. The mouse cursor position and

its speed may be controlled by the expiration

direction and the magnitude. Tooth-touch sound is

utilized for clicking of the mouse. The individual

adaptive circuit functions to learn the individual

characteristics of users such as the magnitude of the

expiration and tooth-touch sound and memorizes the

variation between users. The main signal processing

unit surrounding by a dotted line in this figure, is

constructed on a FPGA chip, SPARTAN-3,

produced by XILINX INC., which was operated at

3V-DC (www.xilinx.com).

4 APPLICATION AS MOUSE

CURSOR POSITION

CONTROLLOR

In this section we applied our device to a mouse

controller called an “Expiration and tooth-touch

sound based mouse”. Figure 10 shows a user fitting

the bone conduction microphone for the tooth-touch

sound and 3-ch piezo film sensors to detect the

expiration signals from three directions, right, center,

and left. We set two targets on a display to use in the

pointing task experiment as shown in Figure 12(a).

After clicking the target A by a tooth-touch, we

moved a mouse cursor from the target A to target B

by our expiration and finally clicked the target B to

finish the trial. The typical characteristic of the

velocity of a mouse cursor is shown in Figure 12 (b).

A peak velocity was observed between the target A

and B. We confirmed that the velocity of a cursor

could be controlled well by the expiration.

Figure 10: Fitting the device.

Figure 11: FPGA based mouse interface.

FPGA chip

PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems

Figure 12: (a) Two target setting on a display (b) Typical

characteristics of the cursor’s velocity.

5 CONCLUSIONS AND

POTENTIAL IMPROVEMENTS

We proposed a novel method for eliminating the

voice and white noise suppression by dyadic wavelet

transform in conjunction with signal adaptive

threshold technique. We showed that our method has

excellent performance at detecting the tooth-touch

sound signal. Next, to improve the usability of

positioning the mouse cursor by the expiration signal,

we modified the control method to adjust the mouse

cursor position more intuitively adapting to the

amplitude of the expiration signal. Finally, we

designed a tooth-touch sound and the expiration

based mouse device circuit using Hardware

Description Language (VHDL) and realized the

system on an FPGA chip in practice.

However, we should further investigate the

details of the expiration operation for the mouse

cursor positioning using the Fitts’ Law, which

describes the relationship between the time to move

the target, the movement distance from the starting

position to the target center, and the target width

(Thompson, 2004). We will confirm whether the

expiration-based mouse pointing follows to the Fitts’

Law and we will design the optimal target size.

Finally, we hope evaluate the usefulness for disabled

persons in more detail.

ACKNOWLEDGEMENTS

This research was partly supported by Grant-in-Aid

for Scientific Research No.21500533.

REFERENCES

Dimitry O. and Gerhard Roth, 2004, Nouse ’use your nose

a mouse’ perceptual vision technology for hands-free

games and interfaces, Image and Vision Computing..

K. Kuzume and T. Morimoto, 2006, Hands-free man-

machine interface device using tooth-touch sound for

disabled persons, Proceedings of 6th International

Conference on Disability, Virtual Reality and

Associate Technology.

K. Kuzume, 2008. A character input system using tooth-

touch sound for disabled people. ICCHP2008, Lecture

Note in Computer Science.

K. Kuzume, 2010. Input device for Disabled Persons

Using expiration and tooth-touch sound signals,

Proceedings of the 25th Annual ACM Symposium on

Applied Computing.

http://www.t-sensor.co.jp/eng/index.html.

D. L. Donoho, 1995. De-noising by soft-thresholding,

IEEE Trans. Inf. Theory.

www.xilinx.com/company/.

http://www.temco-j.co.jp/english.

Thompson, S., Slocum, J. and Bohan, M., 2004. Gain and

angle approach effects on cursor-positioning time with

a mouse in consideration of Fitts’ Low, Human

Factors and Ergonomics Society.

TOOTH-TOUCH SOUND AND EXPIRATION SIGNAL DETECTION AND ITS APPLICATION IN A MOUSE

INTERFACE DEVICE FOR DISABLED PERSONS - Realization of a Mouse Interface Device Driven by Biomedical

Signals