Towards Accurate Browser-based SSVEP Stimuli Generation

Alison Camilleri

1 a

, Chris Porter

1 b

and Tracey Camilleri

2 c

Department of Computer Information Systems, University of Malta, Msida, Malta

Department of Systems and Control Engineering, University of Malta, Msida, Malta

Keywords:

Brain Computer Interface (BCI), Steady State Visually Evoked Potential (SSVEP), Stimuli Generation,

Web-based SSVEP BCI, Human-Computer Interaction (HCI).

Abstract:

Breakthroughs in Brain-Computer Interfacing (BCI) have positively impacted the lives of individuals who

suffer from highly-restrictive physical disabilities. BCIs based on Steady State Visually Evoked Potentials

(SSVEPs) rely on a neuronal response which takes place in the brain’s visual cortex whenever a person fo-

cuses visual attention onto a ﬂickering stimulus. Specialized hardware and software tools exist for stimuli

generation, however little to no empirical evidence exists on the applicability of standard web technologies for

producing accurate and stable stimuli, for use in BCI applications. With the aim of informing efforts for the

development of lightweight, portable and low-cost browser-based BCIs, this paper produces initial evidence

on the performance attained by widely-adopted web technologies, namely CSS and WebGL. Results demon-

strate that for the square wave approximation method, CSS and WebGL are able to effectively render stable

and accurate stimuli on both Google Chrome and Mozilla Firefox.

1 INTRODUCTION

Alternative and Augmentative Communication

(AAC) tools are introduced to assist people who are

unable to communicate due to reduced communica-

tion abilities, generally caused by a variety of motor

or neurological disorders. AAC refers to any form

of aid that can be used to reduce communication

barriers, from simple boards showing symbols of

day-to-day objects or actions, to rudimentary hard-

ware, such as a wand to point to or select actions, as

well as sophisticated eye tracking systems or brain-

computer interfaces (BCIs) (Glennen and DeCoste,

1997). BCIs typically rely on electroencephalogra-

phy (EEG) sensors to capture brain activity, making

them beneﬁcial for individuals with reduced physical

abilities. However, a number of challenges still stand,

particularly concerning accuracy, cost, adoption,

portability and convenience outside a lab setting

(Maggi et al., 2008).

BCIs based on Steady State Visually Evoked Po-

tentials (SSVEPs) are considered to be robust and re-

quire minimal user training to operate. When inter-

acting with an SSVEP-based BCI, electrodes posi-

https://orcid.org/0000-0002-3893-6985

https://orcid.org/0000-0003-3813-2941

https://orcid.org/0000-0002-4908-1863

tioned over the occipital region of the brain register

frequencies equal to that of a ﬂickering stimulus the

user would be attending to. SSVEP detection algo-

rithms are able to determine which stimulus the user

is focusing on, and to utilize this information to exe-

cute a corresponding control function, for example, to

switch between TV channels. In SSVEP-based BCIs,

the accuracy and stability of generated stimuli reﬂects

on the quality and in turn the usability of the interface

(Cecotti et al., 2010).

Literature shows that most SSVEP stimuli gener-

ators are built using desktop-based technologies, such

as C++ (Wang et al., 2010), Psychtoolbox with Mat-

lab (Abbasi et al., 2015), OpenGL (Boyd and Chen,

2012), C# (Lalor et al., 2005), and Java (Hasan et al.,

2015). This kind of software possesses inherent per-

formance guarantees, as it is able to access native re-

sources, without being bound within restrictive run-

times (e.g. browser engine). On the other hand, min-

imal research has been carried out to determine the

applicability of web technologies for building accu-

rate, lightweight, and portable SSVEP stimulation

applications. A few studies have focused on web

browser interaction, but for the most part, these make

use of desktop-based software for stimuli generation,

as in the case of ‘WeBB’ (Yehia et al., 2017). Re-

cent efforts have led to some experimentation with

Camilleri, A., Porter, C. and Camilleri, T.

Towards Accurate Browser-based SSVEP Stimuli Generation.

DOI: 10.5220/0010159400740083

In Proceedings of the 4th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2020), pages 74-83

ISBN: 978-989-758-480-0

Figure 1: 10Hz stimulus presentation using a constant-period approach.

browser-based stimuli generators, including the use

of Graphics Interchange Format (GIF) ﬁles (Saboor

et al., 2018) and Cascading Style Sheets (CSS) (Sa-

boor et al., 2019) to build online spellers. (Rezazadeh

and Sheikhani, 2017) have also succeeded in build-

ing an in-browser stimuli interface, for navigating a

virtual home in V. Realm Builder.

Notwithstanding this, to the best of our knowl-

edge, little to no empirical evidence exists on the

extent to which different web technologies, as well

as underlying browser engines, are able to generate

accurate and stable stimuli for efﬁcient and usable

SSVEP-based interfaces. Furthermore, the viability

of web technologies for presenting on-screen stimuli

using approximation methods, such as square waves

with varied cycle lengths (Wang et al., 2010) has not

yet been explored.

This paper explores this research gap, by investi-

gating the adequacy of standard and widely-available

web technologies, for generating accurate and stable

SSVEP stimuli, which may be adopted in web-based

BCI applications.

2 BACKGROUND

The primary purpose of a BCI system is to extract re-

liable features from brain signals that can be used to

classify users’ intentions accordingly. A BCI fulﬁlls

its purpose in four stages, namely: signal acquisition,

feature extraction, feature translation and device con-

trol or output. The procedure starts off by recording

the electrical activity produced by the brain, typically

done by means of non-invasive scalp electrodes. The

recorded brain signals are very small waves, there-

fore they must be ampliﬁed and digitized before any

further processing can take place. Additionally, the

signal of interest and the background signals must be

discriminated and separated from one another so that

only the relevant signal is analysed. After this process

has been successfully completed, signal characteris-

tics are translated into meaningful commands which

in turn perform some function, such as controlling an

output device (Shih et al., 2012).

This paper considers the use of SSVEPs, which

are resonance phenomena that occur whenever a per-

son focuses attention on a visual stimulus that ﬂick-

ers at a frequency greater than 5Hz. In such a sce-

nario, scalp electrodes positioned over the occipital

region of the brain, register frequencies equal to that

of the ﬂashing stimulus, as well as a number of har-

monic/subharmonic frequencies (Zhu et al., 2010).

LCD and CRT monitors are typically used to ren-

der visual stimuli which are modulated at speciﬁc fre-

quencies.

The success rate of an SSVEP-based BCI is

highly-dependent upon the choice of visual stimulator

(Wang et al., 2008). Having explored various stimuli-

rendering methods, (Cecotti et al., 2010) propound

that the stability of generated stimuli is crucial for

producing effective SSVEP responses. Thus, based

on these assertions, technologies which render stimuli

at stable frequencies should be preferred, as these are

expected to produce better, stronger responses (Ce-

cotti et al., 2010).

As discussed in (Teng et al., 2011), ﬂickering

stimuli are typically presented by means of square

waves having a 50% duty cycle, with a constant pe-

riod throughout (Figure 1 shows such an example for

a 10Hz stimulus). Using this technique, only frequen-

cies which are integer divisors of the screen refresh

rate can be presented in a stable manner. For this

reason, complex applications, for which a substantial

number of targets is required, such as a phone-dialling

program or a speller, cannot present a stimulus for

each possible input. These limitations have hindered

the development of practical BCI applications, result-

ing in decreased Information Transfer Rates (ITRs),

as users are required to perform a greater number of

steps in order to select a desired target (Nakanishi

et al., 2014a).

Alternative methods for stimuli presentation ex-

ist, including the use of triangular, sine, (Teng et al.,

2011) and square waves with varying periods and duty

cycles (Wang et al., 2010). Literature indicates that

Towards Accurate Browser-based SSVEP Stimuli Generation

Figure 2: 11Hz stimulus presentation using the square wave approximation approach.

square waves are able to evoke stronger SSVEP re-

sponses, when compared to sine or triangular waves

(Teng et al., 2011; Chen et al., 2019). Using this tech-

nique, the presentation rate is estimated, such that pe-

riods consist of a varying number of frames in each

cycle (Wang et al., 2010). Stimulus signals may be

computed using the following equation, where f rep-

resents the stimulus frequency, i is the frame index,

and square() generates a square waveform with a 50%

duty cycle:

s( f , i) = square[2π f (i/Re f reshRate)] (1)

In this manner, stimuli frequencies such as 11Hz,

may be presented on a 60Hz monitor, by using peri-

ods of different lengths for each cycle, each consist-

ing of ﬁve or six frames, which correspond to 12Hz

and 10Hz respectively (Wang et al., 2010) (refer to

Figure 2). Thus, the approximation of a speciﬁc fre-

quency, considers two neighbouring frequencies, de-

rived from the constant-period approach (Nakanishi

et al., 2014a). Using this technique, any frequency

lower than half the monitor’s refresh rate can be ap-

proximated (Nakanishi et al., 2014a), thus drastically

increasing the number of concurrent stimuli which

can be displayed on-screen.

In addition to approximation methods, phase-

coding may also be adopted, so that stimuli are dis-

criminated by means of differing phase shifts. (Lee

et al., 2010) demonstrated this technique through

the development of a multi-target, phase-coded BCI,

which presented eight 31.25Hz stimuli using differ-

ent phases. Therefore, through the application of this

technique, stimuli with identical frequencies may be

utilized for building SSVEP stimulation interfaces,

as long as these are coded to have dissimilar phases

(Nakanishi et al., 2014a).

3 AIMS AND OBJECTIVES

As stated by (Saboor et al., 2019), the greatest chal-

lenge stands in determining the most effective tech-

nology for in-browser rendering of stimuli. The ap-

plicability of web technologies, such as Web Graphics

Library (WebGL), Flash or Synchronized Multimedia

Integration Language (SMIL) is not yet known, thus

indicating a gap in research (Saboor et al., 2019). This

work will not make use of SMIL or Flash, due to lack

of support and adoption by major browser vendors.

The de facto technologies used for web graphics and

animation rendering will be considered, namely CSS

and WebGL.

Furthermore, to the best of our knowledge, no re-

search has been performed to determine the efﬁcacy

of approximation techniques within a web browser

context. Provided that these are attainable, a greater

number of concurrent stimuli could be displayed, al-

lowing for the implementation of more complex web-

based SSVEP interfaces.

The aim of this paper is primarily to provide em-

pirical evidence, as to whether SSVEP stimuli can

be accurately generated within a web environment,

considering standard web technologies (i.e. CSS and

WebGL), over two major browsers (Google Chrome

and Mozilla Firefox, which are available across all

platforms). Moreover, this study seeks to determine

whether stimuli produced via square wave approxi-

mations, using standard web technologies, are appli-

cable for generating robust SSVEP responses.

This data will take us one step further towards

building fully ﬂedged, accurate, usable and highly

portable SSVEP-based web browsing interfaces.

The rest of the paper is divided as follows: Section

4 presents the design of the experiment, highlighting

the four developed web-based SSVEP stimulation li-

braries and the main experimental setup. Section 5

discusses the implementation. The results are then

presented in Section 6, while the discussion and con-

clusions are presented in Section 7.

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

4 EXPERIMENT DESIGN

4.1 Overview

For the purpose of this study, constant-period and

square wave approximations were used to build a to-

tal of four SSVEP stimulation applications. Thus,

for both CSS and WebGL, two stimulators were de-

veloped, such that constant-period and approxima-

tion versions were implemented for each selected web

technology. CSS is a client-side language used to en-

hance page content presentation. Browser vendors

build support for CSS based on standardised pub-

lished speciﬁcations. As opposed to CSS, which

operates on the document object model (DOM),

WebGL is speciﬁcally designed for advanced 2D

and 3D graphics rendering on the web, using can-

vas elements, which offer enhanced performance for

graphics-intensive web applications. WebGL is also

widely supported by major browsers, including mod-

ern versions of Google Chrome, Mozilla Firefox, Sa-

fari, Microsoft Edge and Internet Explorer.

The web-based SSVEP stimulation libraries were

tested on a single high-spec machine, using both

Google Chrome and Mozilla Firefox, in order to ob-

serve how their stimuli-rendering performance varies

over time. Throughout the different experimental

runs, data logs for overall CPU, GPU and mem-

ory consumption were recorded. A UNI-T UT372

tachometer was also used to externally measure the

on-screen frequency as produced by the stimuli gen-

erators (refer to Figure 3 for experimental setup). This

was set up in a ﬁxed position with a sampling rate of

255 readings per second. Tachometer data was trans-

ferred over USB to a host computer, making it possi-

ble to determine the accuracy and stability of the gen-

erated stimuli, by means of the selected technologies,

when these were run on different browser engines.

Figure 3: Experimental setup.

4.2 Experimental Procedure

Tests were conducted on a Windows 10 Pro (64-

bit) machine, having an Nvidia GTX1080 GPU, In-

tel Core i7-9700 CPU and 32GB RAM. Stimuli were

presented using an ASUS monitor (version VS228),

running at a 60Hz refresh rate (1920 x 1080 pix-

els). For both CSS and WebGL, two 7-target spellers

were developed, such that each technology was used

for implementing two stimuli generators, using the

constant-period and approximation methods respec-

tively. In this way, stimuli frequencies of 6Hz, 7.5Hz,

9Hz, 10.5Hz, 12Hz, 13.5Hz and 15Hz could be pre-

sented simultaneously on-screen. For the CSS and

WebGL approximation techniques, a total of three

test runs were conducted for ﬁve selected frequen-

cies, namely 7.5Hz, 9Hz, 10.5Hz, 12Hz and 13.5Hz.

Since constant-period methods are unable to gener-

ate frequencies which are not integer divisors of the

screen refresh rate, the 9Hz, 10.5Hz and 13.5Hz fre-

quencies were solely measured for the approxima-

tion approach, described by means of Equation 1.

Tachometer readings for the CSS and WebGL imple-

mentations were recorded on both Google Chrome

(Version 78.0.3904.97 (Ofﬁcial build)(64-bit)) and

Mozilla Firefox (Version 70.0.1 (64-bit)). Both web

browsers were set up to make use of Hardware Accel-

eration, allowing for the majority of graphical inten-

sive tasks to be handled by the GPU.

Each target stimulus frequency was measured for

four minutes by means of an external tachometer (see

Figure 3). This process was repeated three times, such

that for each stimuli-rendering method (i.e. constant-

period and square wave approximations), twelve min-

utes’ worth of data were gathered for every stimulus

frequency, generated via a speciﬁc web technology

and browser combination. For all test runs, a con-

stant distance of 23cm was maintained between the

monitor and tachometer device.

The strength of generated SSVEP responses

greatly depends upon the stability and accuracy of

rendered stimuli. Hence, for each frequency data set,

tachometer values were analyzed in terms of standard

deviation (denotes stimulus stability), as well as dis-

crepancies between mean measured frequencies and

target frequencies (denotes stimulus accuracy). Fur-

thermore, the Independent Samples t-test was applied

to identify any statistically signiﬁcant differences be-

tween two rendering scenarios (for instance, a 7.5Hz

CSS stimulus running on Chrome vs. Firefox, or

a 7.5Hz stimulus running on Chrome generated via

CSS vs. WebGL), in terms of mean measured frequen-

cies. Through this analysis, recommendations could

be made regarding technology-browser combinations

Towards Accurate Browser-based SSVEP Stimuli Generation

which yield higher levels of performance. Finally,

Cohen’s d formula was computed for those instances

where signiﬁcant discrepancies were produced by the

t-test, in order to appropriately quantify the magni-

tude of these dissimilarities.

5 IMPLEMENTATION

To fulﬁll the aims of this study, and as discussed

in earlier sections, a total of four SSVEP stimula-

tion libraries were developed, using CSS and We-

bGL. For each technology, two stimuli-rendering

approaches were considered, namely the constant-

period and square wave approximation methods. The

implemented libraries allow for on-page element ani-

mation, making them suitable for the development of

ﬂexible, modular and customisable SSVEP interfaces.

Prior to stimulus presentation, each stimulator

exploits JavaScript’s requestAnimationFrame()

function to calculate an initial value for the screen re-

fresh rate. In this manner, frame sequence computa-

tions are based on the actual screen refresh rate, thus

ensuring the stability of generated ﬂickers.

For the CSS and WebGL constant-period ap-

proaches, visual ﬂickers were realized by alternating

between dark-coloured and light-coloured frames at a

constant frequency (Nakanishi et al., 2014b). Thus,

taking into consideration a 60Hz monitor, a stimulus

frequency of 10Hz would be represented as follows:

“111000111000111000111000111...”, whereby 1 and

0 represent a dark-coloured and light-coloured frame

respectively. As for the square wave approximation

approach, this made use of Equation 1, to gener-

ate frame sequences for each SSVEP stimulus. In

this case, stimuli frequencies, such as 11Hz, may

be accurately represented on a 60Hz monitor, by

varying the number of frames in a stimulation cy-

cle. Therefore, the square wave approximation tech-

nique, would represent an 11Hz stimulus, by alter-

nating between 5 and 6 frame periods as follows:

“1110001110011100011100111...”.

CSS implementations made use of animation

keyframes for the realization of stimuli ﬂickers. For

the constant-period method, the number of frames in

each stimulation cycle was constant throughout, thus

by calculating the time difference between two con-

secutive frames, animation duration for stimulus cy-

cles could be set, and used to alter stimulus state at a

regular interval.

For the square wave approximation approach,

stimuli ﬂickers are also produced by means of

CSS keyframes, which are dynamically-generated

via JavaScript, prior to the rendering of stimuli.

Keyframes are set up to alter stimulus opacity, based

on values produced by Equation 1. In order to set

up the keyframe, square wave sequences are pre-

computed for the required number of frames. For

our implementation, the ‘required number of frames’

value is determined based on the frame index, at

which the square wave cycle repeats itself.

In order to improve rendering efﬁciency, all CSS

keyframes were set up to make changes to ele-

ments’ ‘opacity’ property, so that stimulus presenta-

tion would involve less rendering stages. Accord-

ing to (Lewis, n.d.), the most performant version of

the pixel pipeline eliminates the need for constant re-

paints, and only requires compositing changes. By

promoting a speciﬁc element to its own layer, the

‘opacity’ property can be handled by the compositor

alone, thus reducing the number of steps involved in

rendering visual ﬂickers.

The WebGL stimulation implementations make

use of a single, large canvas for presenting on-screen

SSVEP stimuli. Since browsers impose a limit on

the number of concurrent WebGL contexts, the use

of a canvas element for each stimulus was not consid-

ered to be a viable option. By using a canvas which

covers the entire window, placeholder <div> ele-

ments can be used to set stimuli locations. Through

the use of element.getBoundingClientRect(),

the viewport is set to cover the <div>’s loca-

tion, so that stimulus ﬂickers may be generated

within the speciﬁed area. Prior to the genera-

tion of on-screen stimuli, two textures are rendered

for each stimulus; a dark-coloured texture, and a

light-coloured texture. During stimulus presenta-

tion, WebGLRenderingContext.bindTexture() is

invoked to display dark/light textures at speciﬁc time

intervals.

For WebGL’s constant-period approach, the frame

sequence was computed for a single period, and

stored within an array, so that it could be repeatedly

used for stimulus presentation. In this way, for ev-

ery requestAnimationFrame() invocation, a spe-

ciﬁc frame value is read out of the array, to display

the appropriate texture. For instance, a frame value of

0 would indicate that a black texture is to be shown,

while a frame value of 1 would signify white texture

presentation.

A similar approach was also adopted for WebGL’s

approximation technique, however, as discussed pre-

viously, square wave sequence values were computed

via Equation 1, for the required number of frames.

The resultant values were also stored within an array,

and looped over continuously to display appropriate

stimuli ﬂickers.

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

Table 1: Table showing results for the square wave approximation method, including the average bias and standard deviation

(calculated for 3 test runs) for each technology, browser, and stimulus frequency combination.

Technology Web Browser Actual Freq. (Hz) Mean Measured Freq. (Hz) Bias Standard Deviation

CSS Chrome 7.5 7.49982786 0.00017214 0.000357508

CSS Chrome 9 8.999826161 0.000173839 0.000033826

CSS Chrome 10.5 10.510888396 0.010888396 0.016871103

CSS Chrome 12 11.999742734 0.000257266 0.00086882

CSS Chrome 13.5 13.499745745 0.000254255 0.021762828

Mean 0.002349179 0.007978817

WebGL Chrome 7.5 7.499837479 0.000162521 0.000752463

WebGL Chrome 9 8.999831297 0.000168703 0.0000183087

WebGL Chrome 10.5 10.511127822 0.011127822 0.016984468

WebGL Chrome 12 11.999758494 0.000241506 0.000930272

WebGL Chrome 13.5 13.50000045 0.000000445 0.022015793

Mean 0.002340199 0.008140261

CSS Firefox 7.5 7.500371427 0.000371427 0.003879366

CSS Firefox 9 9.000257754 0.000257754 0.003808133

CSS Firefox 10.5 10.510751762 0.010751762 0.017077129

CSS Firefox 12 12.000036605 0.000036605 0.003783792

CSS Firefox 13.5 13.500389542 0.000389542 0.023047437

Mean 0.002361418 0.010319171

WebGL Firefox 7.5 7.499832556 0.000167444 0.0000197006

WebGL Firefox 9 8.999830261 0.000169739 0.0000224204

WebGL Firefox 10.5 10.511080384 0.011080384 0.017025472

WebGL Firefox 12 11.998369124 0.001630876 0.02129115

WebGL Firefox 13.5 13.499634118 0.000365882 0.02178422

Mean 0.002682865 0.012028593

6 RESULTS

6.1 Stimuli Accuracy and Stability

In the context of an SSVEP-based BCI, stimuli stabil-

ity and accuracy are crucial for the generation of ef-

fective SSVEP responses. Hence, for the purpose of

this analysis, technology and browser combinations

are considered to be highly performant, if they render

stimuli at frequencies which are very close to their

expected values, with minimal variance over time.

Thus, to determine the applicability of web tech-

nologies for efﬁcacious generation of SSVEP re-

sponses, tachometer data was initially analyzed in

terms of (a) dispersion of values (denotes stimulus

stability), as well as (b) discrepancies between mean

measured frequencies and target frequencies (de-

notes stimulus accuracy). For each stimuli-rendering

method, technology and browser combination, both

mean and standard deviation values were computed,

for stimulus frequency measurements captured by the

tachometer. This procedure was repeated for fre-

quency data sets pertaining to a single test run, as well

as amalgamated data sets, consisting of frequency val-

ues for the three separate runs.

Mean values for merged data sets were subse-

quently used to compute a value for the ‘bias’, which

is deﬁned as the absolute value of the mean measured

frequency, subtracted from the target frequency (re-

fer to Equation 2). The bias value indicates the extent

to which mean measured frequencies differ from their

expected values.

Bias = |Target Frequency − Mean Frequency | (2)

For both constant-period and approximation

methods, average values for the bias and standard de-

viation were also calculated, taking into considera-

tion all stimuli frequencies, generated via a speciﬁc

technology and browser combination (refer to Table 1

for square wave approximation results). These values

may be considered as performance indicators, which

demonstrate the technology and browser’s ability to

generate stimuli at stable and accurate frequencies.

Stimuli generated via constant-periods were found

to make use of the exact frame sequences as square

wave approximations, for presenting stimuli at 7.5Hz

and 12Hz (integer divisors of the screen refresh rate).

Hence, results in terms of bias and standard deviation

did not yield signiﬁcant discrepancies across these

two rendering approaches. For this reason, this paper

shall focus on discussing the applicability of distinct

web technology and browser permutations, for pre-

senting stable and accurate stimuli via square wave

approximations.

From results obtained, it is evident that high per-

formance levels were attained for each target fre-

Towards Accurate Browser-based SSVEP Stimuli Generation

Table 2: Summary of t-test and Cohen’s d formula results (calculated for the amalgamation of 3 test runs) for stimuli generated

via the square wave approximation method, under the various rendering scenarios.

Stimulus Comparison P-Value (T-Test) Cohen’s d Value

CSS 7.5Hz Chrome vs. Firefox 0 0.19732

CSS 9Hz Chrome vs. Firefox 0 0.16027

CSS 10.5Hz Chrome vs. Firefox 0.881 -

CSS 12Hz Chrome vs. Firefox 0 0.10705

CSS 13.5Hz Chrome vs. Firefox 0.184 -

WebGL 7.5Hz Chrome vs. Firefox 0.666 -

WebGL 9Hz Chrome vs. Firefox 0.016 0.05062

WebGL 10.5Hz Chrome vs. Firefox 0.827 -

WebGL 12Hz Chrome vs. Firefox 0 0.09220

WebGL 13.5Hz Chrome vs. Firefox 0.424 -

Chrome 7.5Hz CSS vs. WebGL 0.442 -

Chrome 9Hz CSS vs. WebGL 0 0.18884

Chrome 10.5Hz CSS vs. WebGL 0.317 -

Chrome 12Hz CSS vs. WebGL 0.386 -

Chrome 13.5Hz CSS vs. WebGL 0.59 -

Firefox 7.5Hz CSS vs. WebGL 0 0.19644

Firefox 9Hz CSS vs. WebGL 0 0.15875

Firefox 10.5Hz CSS vs. WebGL 0.35 -

Firefox 12Hz CSS vs. WebGL 0 0.10905

Firefox 13.5Hz CSS vs. WebGL 0.114 -

quency, technology, and browser combination, as sig-

niﬁed by the remarkably small mean values achieved

for the bias and standard deviation throughout (refer

to Table 1). Results show that the largest inaccura-

cies/instabilities were noted when the WebGL stim-

uli generator was run using Firefox, resulting in mean

bias and standard deviation values of 0.002682865

and 0.01202859252 respectively. Although this tech-

nology and browser combination resulted in the high-

est level of bias and variance, it is worth noting that

such degrees of inaccuracy and instability are very

small, and are thus considered to be negligible in the

context of an SSVEP BCI system. Therefore, stim-

uli generated under such rendering conditions should

still be able to evoke appropriate SSVEP responses.

6.2 Performance Comparisons

Based on the Central Limit Theorem (CLT), the dis-

tribution of sample means approaches a normal dis-

tribution as the sample grows in size. Irrespective of

whether the source population is normal or skewed,

this holds true, provided that the sample size is ≥30

and sampling is carried out by replacement (LaMorte,

2016). For a speciﬁc stimulus frequency, rendered via

a speciﬁc technology-browser permutation, tachome-

ter data for the amalgamation of three test runs con-

sisted of ≈ 4000 data points. Hence, based on CLT

criteria, these data sets are considered to have sufﬁ-

ciently large sample sizes, which indicate that data

should follow a normal distribution. As a result, para-

metric tests, such as the Independent Samples t-test

may be applied to make the necessary inferences.

Since the t-test assumes homogeneity of variance, p-

values produced by the Levene’s test were initially

computed, and used to reference the appropriate t-

test result. SPSS produces two p-values for the t-test

statistic, which were quoted based on whether the as-

sumption for equality of variances was violated or not

(KentStateUniversity, 2019).

The t-test was executed for distinct technology-

browser combinations, each time making use of two

frequency data sets, consisting of 12 minutes’ worth

of tachometer data (pertaining to 3 test runs). The

level of signiﬁcance was taken to be 0.05 for all tests,

thus p<.05 indicated statistically signiﬁcant discrep-

ancies between stimuli generation technologies, run-

ning on different browsers.

Results produced by the Independent Samples t-

test are listed in Table 2, to showcase any statisti-

cally signiﬁcant discrepancies between two distinct

rendering scenarios (for instance, a 7.5Hz CSS stim-

ulus running on Chrome vs. Firefox, or a 7.5Hz

stimulus running on Chrome generated via CSS vs.

WebGL). As evidenced in Table 1, all technology-

browser combinations fared relatively well, and were

able to produce stimuli which remained close to their

target frequencies, with minimal deviation over time.

Through the application of the t-test, the degree by

which these rendering setups differed from one an-

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

other could be further understood. This test was con-

sidered to be a ﬁrst pass analysis, for the provision of

recommendations, regarding technology and browser

setups, which should be favoured when building web-

based SSVEP stimuli generators.

For stimuli rendered via square wave approxima-

tions, it was observed that when running the CSS

stimuli generator on both Chrome and Firefox, a to-

tal of three signiﬁcant discrepancies were noted in

stimuli-rendering performance. In this case, bias val-

ues were very close to one another, however Firefox

showed an overall higher variance (Table 1), which in-

dicates that Chrome might be a better alternative for

presenting CSS-generated stimuli. In the case of We-

bGL, similar observations were made, whereby cal-

culated bias values were very close to one another,

yet stimuli generated via Firefox had a greater de-

gree of instability. Thus, results achieved are con-

sistent for both technologies, and favour the use of

Chrome over Firefox, for presenting stable and accu-

rate SSVEP stimuli.

When running both the CSS and WebGL stim-

uli generators on Chrome, results obtained were very

similar, both in terms of bias and standard deviation

(Table 1). Thus, both technologies are considered to

be comparable for effective presentation of SSVEP

stimuli via Chrome. However, when running the same

stimuli generators on Firefox, three out of ﬁve fre-

quencies were found to be signiﬁcantly different, with

a p-value less than 0.05 (Table 2). In this scenario,

WebGL had an overall higher bias and standard de-

viation when compared to CSS, which indicates that

when using Firefox, CSS might be a better alternative.

In the context of an SSVEP-based BCI, such mi-

nor discrepancies are not considered to be of signiﬁ-

cance, as all stimuli frequencies remained very close

to their expected values, and showed minimal vari-

ance over time. This variance, which was less than

0.03 across all cases, shows that these stimuli ren-

dering technologies are adequate, even for complex,

multi-target systems, such as the 40-target BCI devel-

oped by (Nakanishi et al., 2017), where the frequency

difference between stimuli was as low as 0.2Hz.

As stated by (Halsey, 2019), p-values provide lim-

ited information about our data, and can thus be mis-

interpreted. P-values tend to exhibit high sample-to-

sample variability, which does not reliably indicate

the amount of evidence against a speciﬁed null hy-

pothesis. Thus, these values should only be used for

providing ﬁrst pass evidence, about a phenomenon

being studied (Halsey, 2019).

As opposed to signiﬁcance testing, estimation

statistics are targeted towards determining the mag-

nitude of an effect and its precision. An effect size

may simply be described as the “the degree to which

the null hypothesis is false” (Enzmann, 2015). A

well-known measure of effect size is Cohen’s d for-

mula, which compares the means of two independent

groups.

For the Independent Samples t-test, Cohen’s d

value may be determined by calculating the mean dif-

ference between two groups, and then dividing this

result by the pooled standard deviation. Equation 3

depicts Cohen’s d formula (Rosenthal et al., 1994),

whereby M

and M

refer to the means of the ﬁrst and

second samples, and SD

pooled

is the pooled standard

deviation of the two samples:

d =

− M

pooled

(3)

Equation 4 is used to compute SD

pooled

, whereby SD

and SD

represent the standard deviation for the ﬁrst

and second samples respectively:

pooled

− SD

(4)

For the purpose of this study, Cohen’s d formula

was computed to determine the effect size, for in-

stances where signiﬁcant discrepancies were noted by

the t-test (refer to Table 2). When considering large

sample sizes, minor effects, such as minute discrep-

ancies between means, can result in signiﬁcant statis-

tical differences (Walker, 2008). Thus, by making use

of Equation 3, the magnitude of differences between

two groups can be appropriately quantiﬁed.

Results obtained for Cohen’s d value (Table 2)

were interpreted based on commonly used bench-

marks (Lakens, 2013), whereby d values equal to 0.2,

0.5, and 0.8, denote small, medium and large effect

sizes respectively. Thus, if discrepancies between two

group means are smaller than 0.2 standard deviations,

differences are considered to be negligible, even if

statistically signiﬁcant (Walker, 2008).

Based on this, values produced by Cohen’s d for-

mula (refer to Table 2) indicate that despite signif-

icant differences produced by the t-test, these dis-

crepancies correspond to a small effect size (d<0.2).

Thus, for all permutations listed in Table 2 (refer

to the ‘Stimulus’ and ‘Comparison’ columns), the

means for stimuli frequencies generated under the

various rendering scenarios are similar, and any dis-

crepancies are considered to be negligible. In prac-

tice, results attained demonstrate that irrespective of

the selected technology-browser combination, appro-

priate SSVEP responses may be evoked from web-

Towards Accurate Browser-based SSVEP Stimuli Generation

based stimuli generated via square wave approxima-

tion methods.

7 CONCLUSIONS

This study provides evidence-based arguments on the

applicability of standard browser-based technologies

for SSVEP stimuli generation. Results show that CSS

and WebGL may be used to render effective SSVEP

stimuli, on both Google Chrome and Mozilla Firefox.

This was demonstrated by the consistent stability and

accuracy of the generated stimuli.

Furthermore, this study successfully adopted the

square wave approximation technique, for presenting

stimuli frequencies which are non-integer divisors of

the screen refresh rate. Using this method, a greater

number of concurrent on-screen stimuli can be dis-

played, thus allowing for the development of complex

BCI applications, such as web-based spellers, with a

target for each possible input.

Contrary to state of the art SSVEP stimulation

technologies, in-browser stimuli generators guaran-

tee cross-platform portability, while further lowering

barriers for BCI-enabled web development. Building

a web-based BCI also depends on an ecosystem of

technologies, bringing forth various challenges; from

efﬁcient and cost-effective user-side setup, to remote

signal processing, and ﬁnally, real-time browser con-

trol.

ACKNOWLEDGEMENTS

We would like to thank Dr Sandro Spina from the

Computer Graphics and Visualisation Group, for pro-

viding insights, as well as state of the art hardware,

on which tests were carried out. We would also like

to thank Rosanne Zerafa from the Biomedical Cyber-

netics lab, for insights provided at the initial stages of

this study.

REFERENCES

Abbasi, M. A., Gaume, A., Francis, N., Dreyfus, G., and

Vialatte, F.-B. (2015). Fast calibration of a thirteen-

command bci by simulating ssveps from trains of tran-

sient veps-towards time-domain ssvep bci paradigms.

In Neural Engineering (NER), 2015 7th International

IEEE/EMBS Conference on, pages 186–189. IEEE.

Boyd, J. and Chen, Y. (2012). An open source stimulator

for ssvep-based bcis. In Proceedings of the 50th An-

nual Southeast Regional Conference, pages 124–129.

ACM.

Cecotti, H., Volosyak, I., and Gr

aser, A. (2010). Reliable

visual stimuli on lcd screens for ssvep based bci. In

Signal Processing Conference, 2010 18th European,

pages 919–923. IEEE.

Chen, X., Wang, Y., Zhang, S., Xu, S., and Gao, X.

(2019). Effects of stimulation frequency and stimula-

tion waveform on steady state visual evoked potentials

using computer monitor. Journal of Neural Engineer-

ing.

Enzmann, D. (2015). Notes on effect size measures for the

difference of means from two independent groups: the

case of cohen’sd and hedges’g. January, 12:2015.

Glennen, S. and DeCoste, D. C. (1997). The handbook of

augmentative and alternative communication. Cen-

gage Learning.

Halsey, L. G. (2019). The reign of the p-value is over: what

alternative analyses could we employ to ﬁll the power

vacuum? Biology letters, 15(5):20190174.

Hasan, M. K., Hossain, T. M., Faisal, A. K., and Ahmad,

M. (2015). Effect of diverse stimulations of brain on

eeg signal for ssvep based brain computer interface.

In 2015 International Conference on Electrical Engi-

neering and Information Communication Technology

(ICEEICT), pages 1–6. IEEE.

KentStateUniversity (2019). SPSS Tutorials: Independent

Samples T Test. Retrieved Nov 17, 2019, from https:

//libguides.library.kent.edu/SPSS/IndependentTTest.

Lakens, D. (2013). Calculating and reporting effect sizes

to facilitate cumulative science: a practical primer for

t-tests and anovas. Frontiers in psychology, 4:863.

Lalor, E. C., Kelly, S. P., Finucane, C., Burke, R., Smith, R.,

Reilly, R. B., and Mcdarby, G. (2005). Steady-state

vep-based brain-computer interface control in an im-

mersive 3d gaming environment. EURASIP Journal

on Advances in Signal Processing, 2005(19):706906.

LaMorte, W. (2016). Central Limit Theo-

rem. Retrieved Nov 17, 2019, from http:

//sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/

BS704 Probability/BS704 Probability12.html.

Lee, P.-L., Sie, J.-J., Liu, Y.-J., Wu, C.-H., Lee, M.-H.,

Shu, C.-H., Li, P.-H., Sun, C.-W., and Shyu, K.-K.

(2010). An ssvep-actuated brain computer interface

using phase-tagged ﬂickering sequences: a cursor sys-

tem. Annals of biomedical engineering, 38(7):2383–

2397.

Lewis, P. (n.d.). Stick to Compositor-Only Properties and

Manage Layer Count. Retrieved Nov 15, 2019, from

https://developers.google.com/web/fundamentals/

performance/rendering/stick-to-compositor-only-

properties-and-manage-layer-count.

Maggi, L., Parini, S., Perego, P., and Andreoni, G. (2008).

BCI++: an object-oriented BCI prototyping frame-

work. Citeseer.

Nakanishi, M., Wang, Y., Chen, X., Wang, Y.-T., Gao, X.,

and Jung, T.-P. (2017). Enhancing detection of ssveps

for a high-speed brain speller using task-related com-

ponent analysis. IEEE Transactions on Biomedical

Engineering, 65(1):104–112.

Nakanishi, M., Wang, Y., Wang, Y.-T., Mitsukura, Y., and

Jung, T.-P. (2014a). Generating visual ﬂickers for elic-

CHIRA 2020 - 4th International Conference on Computer-Human Interaction Research and Applications

iting robust steady-state visual evoked potentials at

ﬂexible frequencies using monitor refresh rate. PloS

one, 9(6):e99235.

Nakanishi, M., Wang, Y., Wang, Y.-T., Mitsukura, Y., and

Jung, T.-P. (2014b). A high-speed brain speller using

steady-state visual evoked potentials. International

journal of neural systems, 24(06):1450019.

Rezazadeh, Z. and Sheikhani, A. (2017). An ssvep-based

brain-computer interface to navigate in a virtual home.

In Proceedings of the 7th International Conference on

Bioscience, Biochemistry and Bioinformatics, pages

22–27. ACM.

Rosenthal, R., Cooper, H., and Hedges, L. (1994). Paramet-

ric measures of effect size. The handbook of research

synthesis, 621:231–244.

Saboor, A., Benda, M., Gembler, F., and Volosyak, I.

(2019). Word prediction support model for ssvep-

based bci web speller. In International Work-

Conference on Artiﬁcial Neural Networks, pages 430–

441. Springer.

Saboor, A., Gembler, F., Benda, M., Stawicki, P., Rezeika,

A., Grichnik, R., and Volosyak, I. (2018). A browser-

driven ssvep-based bci web speller. In 2018 IEEE In-

ternational Conference on Systems, Man, and Cyber-

netics (SMC), pages 625–630. IEEE.

Shih, J. J., Krusienski, D. J., and Wolpaw, J. R. (2012).

Brain-computer interfaces in medicine. In Mayo

Clinic Proceedings, volume 87, pages 268–279. El-

sevier.

Teng, F., Chen, Y., Choong, A. M., Gustafson, S., Reich-

ley, C., Lawhead, P., and Waddell, D. (2011). Square

or sine: Finding a waveform with high success rate of

eliciting ssvep. Computational intelligence and neu-

roscience, 2011:2.

Walker, I. (2008). Null hypothesis testing and ef-

fect sizes. Retrieved Nov 17, 2019, from

https://people.bath.ac.uk/pssiw/stats2/page2/page14/

page14.html.

Wang, Y., Gao, X., Hong, B., Jia, C., and Gao, S. (2008).

Brain-computer interfaces based on visual evoked po-

tentials. IEEE Engineering in medicine and biology

magazine, 27(5):64–71.

Wang, Y., Jung, T.-p., et al. (2010). Visual stimulus

design for high-rate ssvep bci. Electronics letters,

46(15):1057–1058.

Yehia, A. G., Eldawlatly, S., and Taher, M. (2017). Webb:

A brain-computer interface web browser based on

steady-state visual evoked potentials. In Computer

Engineering and Systems (ICCES), 2017 12th Inter-

national Conference on, pages 52–57. IEEE.

Zhu, D., Bieger, J., Molina, G. G., and Aarts, R. M. (2010).

A survey of stimulation methods used in ssvep-based

bcis. Computational intelligence and neuroscience,

2010:1.

Towards Accurate Browser-based SSVEP Stimuli Generation