MusicBlocks: An Innovative Tool for Learning the Foundations of Music

Beatrice Miotti

, Luca Bassani, Enrico Cauteruccio and Marco Morandi

INDIRE, Istituto Nazionale di Documentazione ed Innovazione e Ricerca Educativa, via Buonarroti 5, Florence, Italy

Keywords:

Music Teaching, Laboratory Teaching, Open-source, Lego

.

Abstract:

The Italian National Guidelines (Miur, 2012) and in particular the legislative decree no. 60 highlight the

competencies in the artistic and expressive ﬁeld which students must acquire by the end of the ﬁrst cycle of

education. Indire in collaboration with the Italian Ministry of Education started research activities in order to

support music teachers in the teaching activities and in the implementation of best practices in their classes.

The research presented in this paper results from listening to teachers’ needs. It describes the ideation, reali-

sation and future developments of the experimentation of MusicBlocks: a tool patented by the research group,

which enables tangible music production, overcoming the difﬁculties deriving from learning how to play a

musical instrument. MusicBlocks is an easy-to-use tool due to its structure especially conceived to be used by

students as a stand-alone tool. Students can compose their own melody and listen to its execution by laying

Lego

 bricks. If this does not provide practical help to learn how to play an instrument, it surely helps to

acquire the propaedeutic competences: rhythm, melody, and harmony, which are necessary in order to develop

an interest in playing a musical instrument. Experimentations with MusicBlocks are currently undergoing in

lower secondary schools, receiving a high degree of interest and appreciation from students and teachers.

1 INTRODUCTION

According to the “National guidelines for the curricu-

lum of infant schools and the ﬁrst cycle of education”

(Miur, 2012), some of the basic concepts of founda-

tions of music: rhythm, melody, and harmony can be

studied and understood since the ﬁrst cycle of educa-

tion. Clearly, these are abstract concepts that can be

exempliﬁed only by direct participation and experi-

ence. In this context, supported by a wide literature

about hands-on activities (Papert, 1984) and ludic di-

dactics (Rogers, 1973), the development and design

of an innovative device for the learning of music con-

cepts has been carried out. In the ﬁrst part we are go-

ing to present the theoretical basis of active and mu-

sic didactics and an excursus on the regulations and

initiatives promoted by the research group in collab-

oration with the Italian Ministry of Education for the

acquisition of skills in the ﬁeld of music and the pro-

motion of innovative teaching methods. In the second

part we will deal with the development and design of a

device called MusicBlocks. Through a simple and in-

teractive interface it enables students to compose their

own melody without the need to be able to play mu-

https://orcid.org/0000-0001-7382-8747

https://orcid.org/0000-0002-7948-4380

sical instruments, giving them the possibility to focus

on the theoretical basis underlying music. Therefore,

the strength of MusicBlocks is its capacity to make

musical composition visual, which helps to become

acquainted with the fundamental principles of music

in an intuitive and engaging manner.

1.1 Laboratory Teaching as a Method

to Develop Music Competences

Referring to Piaget’s theory of cognitive develop-

ment already at nursery school and in the ﬁrst years

of primary school, children are able to use symbols

and refer to symbolic play as a tool to socialize and

represent the world. They are therefore capable of

substituting an object to represent other things and

their playful activity is enriched with symbolic ad-

dition capacities. According to Piaget, older chil-

dren, up to lower secondary school, in the “concrete

operational” stage, show the ability to operate logi-

cally on previously acquired symbolic mental struc-

tures, and what is more, they also acquire the princi-

ple of reversibility so that it is possible to go back

from certain actions with the same but inverse op-

erations. In this sense, it seems important to con-

sider the practical and theoretical structure of con-

Miotti, B., Bassani, L., Cauteruccio, E. and Morandi, M.

MusicBlocks: An Innovative Tool for Learning the Foundations of Music.

DOI: 10.5220/0011152100003182

In Proceedings of the 14th International Conference on Computer Supported Education (CSEDU 2022) - Volume 1, pages 475-484

ISBN: 978-989-758-562-3; ISSN: 2184-5026

475

structivism that originates from Piaget’s works, ac-

cording to which knowledge is actively constructed

by the learner and cannot be passively transmitted by

the teacher. The concept of “learning by doing” is es-

pecially powerful. This is justiﬁed in Piaget’s studies,

according to which formal thinking, acquired from

the age of 11 onwards, retraces the stages of concrete

thought, which is characteristic of the previous cogni-

tive stage, but at a hypothetical-deductive, non-verbal

level. From this analysis emerges that the concrete

operations are a previous conquest with respect to the

hypothetical-inductive operations, and therefore more

deeply rooted in the individual (Valentini and Tallan-

dini, 1998). In 1984, Papert proposed an evolution of

Piaget’s constructivism according to which the learn-

ers themselves contribute to building concepts by in-

teracting with appropriate materials, called cognitive

artefacts, which, used in a cooperative environment

under the guidance of the teacher, facilitate learning

(Papert, 1984). The manipulation and use of artefacts

enable to face up to the problem of the contextualiza-

tion of knowledge (Merlo, 2017). As further conﬁr-

mation of what stated above, in the 1960s, cognitive

didactics developed, and Ausubel’s theory of “mean-

ingful learning” was developed (Ausubel, 1968). Ca-

puano and colleagues, describe the effect of a deep

learning “In short, meaningful learning enables stu-

dents to become strategic, through the development

of skills of a metacognitive (learning to learn), re-

lational (knowing how to work in a group) or atti-

tudinal nature (autonomy and creativity)” (Capuano

et al., 2018, p. 9). The meaningful construction of

knowledge occurs when students are placed in a posi-

tion to experiment, move autonomously within a dis-

cipline, act together in a relationship of mutual ex-

change, shared reﬂection, dialogue and negotiation

with others (Jonassen et al., 2007).

1.2 Ludic Didactics

Playing and its more scientiﬁc and pedagogical

framework as ludic didactics is one of the ways in

which “meaningful learning” takes place (Rogers,

1973; Cairo, 2012; Rizzo, 2014). With this term we

intend to refer to holistic learning (that is, investing

the affective and emotional dimension of the subject,

not only the knowledge sphere), which is strongly

based on experience and capable of stimulating the

learner’s interests. Being this kind of learning di-

rectly linked to the learner’s interests, it is also self-

motivated and self-evaluated: the interest is in play-

ing itself and the objective is to play. Ilaria Sudati

presents the application of the ludic theory to Ital-

ian language teaching and learning in adults (Sudati,

2014). Caon and Rutka deﬁne the ludic method as, in

essence, able to create meaningful learning, by stimu-

lating aspects of motivation, learning by doing, col-

laboration and implicitly autonomy, reﬂection, and

cultural difference (Caon and Rutka, 2004). Krashen,

as reported in (Sudati, 2014), states that “one learns a

language better if he/she forgets that he/she is learn-

ing” (Krashen, 1983, p. 213). Making a parallel be-

tween language and music is easy, as well as regard-

ing the variables in their learning process: language

has the alphabet, music has notes; language is written

according to syntactic and logical rules, music has a

grammar made up of the duration of notes and beats.

Therefore, the same considerations expressed before

ﬁnd a correspondence in the learning style of music;

and the ludic component and learning by playing are

relevant also in this context.

1.3 The Evolution of Music Teaching

The 1980s saw important changes in the ﬁeld of mu-

sic pedagogy and didactics. An innovation in teach-

ing, focusing on creativity, enhancing the musical im-

provisational and compositional act as fundamental

for an organic development of musicality and skills

can be ﬁnd in (Delfrati, 1979; Porena, 1979; Piazza,

1979; Piazza, 1984; Addessi et al., 1996). Music ed-

ucation was seen from different angles, such as the

linguistic connotation (Della Casa, 1985), semantic-

cultural (Stefani, 1982), and semantic-communicative

perspectives (Baroni, 1978). In (Addessi and Piras,

2020) a recent analysis of the evolution of teaching

starting from the 1980s is reported. More recently

Enrico Bottero and Irene Carbone proposed Bianchi’s

experience and his vision of creativity and didactics

of music that “aim to form the bodily, sensorial and

perceptive foundations of basic education. They aim

to strengthen the pre-categorical foundations that are

also useful for other school disciplines and activities:

the learning of reading and writing, linguistic expres-

sion, mathematics, etc.” (Bottero and Carbone, 2003,

p. 15).

1.4 Application of Ludic Tools (Lego) to

the Didactics of Music

In the 90’s, on the basis of Piaget’s constructivist the-

ory, Papert’s constructionism and Frank R. Wilkons’s

hand-mind relationship, Lego proposed the method

Lego Serious Play, a transversal method which start-

ing from a real problem guides students, but also

adults, in the construction of a narrative experience

which leads to the personal solution of the problem.

The Lego bricks work as a catalyst, as a tool to “think

CSME 2022 - 3rd International Special Session on Computer Supported Music Education

476

through your hands” which is in fact the slogan of

the method. Thus, we ﬁnd examples of the use of

bricks for the study of science and physics in (Camp-

bell et al., 2011; B

orner, 2007; Celli and Gonella,

2015). In (Salmaso, 2013) and (Salmaso, 2014) ex-

periences in primary school - where Lego bricks have

been used for the development of visual-spatial and

planning skills, also in an inclusive and gender-equal

perspective - are presented. Lego bricks are then rou-

tinely used in coding and educational robotics activi-

ties, while online different experiences of using bricks

in grammar and logical analysis are told, as personal

initiatives of teachers

, as well as in the animation of

stories and storytelling

. Oestermaier and colleagues

presented an application for an interactive multi-touch

table, which exploiting transparent grids and Lego

bricks enables children to create a music composi-

tion: the “LEGO music learning composition with

bricks” (Oestermeier et al., 2015). A software ap-

proach which encourages computational thinking was

2017; Barat

e et al., 2017). They developed a “Mul-

timodal LEGO - Based learning activity” where ele-

ments of the domain of music and information tech-

nology are mixed together. The learning activity in-

cludes also a web application to support this approach

which foresees the disposition of bricks in a delim-

ited area, including also the third dimension, in order

to organise multiple melodic lines. Matthew Shifrin,

(Mattheij, 2022), a visually-impaired English musi-

cian and singer, has devised a methodology for the

study of rhythm, pitch and duration of sounds which

involves the use of Lego bricks arranged on a plate

of variable dimensions, used as an alternative stave in

which the abscissa represents the passage of time and

the ordinate the pitch of the sounds. The dimensions

associated with the colours of the bricks represent the

durations of the sounds.

1.5 Guidelines and Ministerial

Initiatives

Since 2012, Indire (National Institute for Documen-

tation, Innovation and Educational Research), a re-

search body of the Italian Ministry of Education, has

conducted research activities in the ﬁeld of music edu-

cation, through a series of projects in order to promote

the dissemination of musical practice in schools of all

levels: in particular vocal and instrumental practice

for the ﬁrst cycle of education, new technologies, and

https://portalebambini.it/analisi-logica-con-i-lego/

and https://youtu.be/T2LdyYOdfoY

https://codingerobotica.indire.it/uploads/

CODINGEROBOTICA/CONTRIBUTI/galati 1.pdf

music for upper and lower secondary schools, docu-

mentation and dissemination of best practices on mu-

sic teaching and skills related to the ﬁeld of music

listening. In the context of laboratory teaching and

innovation of the curriculum in the artistic-expressive

area, in recent years, Indire has achieved a series of

signiﬁcant experiences in terms of commitment and

results: from the documentation and sharing environ-

ment “Music at school”

to the 31st National Music

Festival “Music unites schools”

; from the imagina-

tion and development of innovative prototypes to the

broader integration of music education in the context

of the plan for the arts. The guidelines for experi-

mentation led by the Ministry of Education have been

clearly expressed in a series of legislative acts that

have promoted the prospect of the gradual inclusion

of musical practice in the core curriculum of all stu-

dents: the Decree of the Minister of Education no.

8 of 31 January, 2011

, and the national guidelines

for the curriculum of infant schools and the ﬁrst cycle

of education. More recently, the Legislative Decree

of 13 April, 2017 no. 60, “Regulations on the pro-

motion of humanistic culture, on the enhancement of

cultural heritage and productions, and on the support

of creativity”

, introduced the “Themes of creativity”

as fundamental components of school knowledge, of

the wealth of intellectual knowledge of every human

being, and therefore of the curriculum itself. They

make use of the help of technological innovation and

involve four areas: music and dance; theatre and per-

formance; arts and visual arts; language and creativ-

ity.

2 RESEARCH FRAMEWORK

Starting from what Shifrin proposed in his research,

i.e. the use of a Lego plate and bricks as placehold-

ers for the notes, Indire conducted a study and tech-

nological research activity for the implementation of

a computer-based tool that could automatically re-

produce the melody starting from the reading of the

plate duly populated with bricks. The research idea

that guided the development of MusicBlocks derives

from the analysis of the literature (as indicated in

Section 1.1) and from the considerations that arise

with respect to learning methods in collaborative and

manipulative contexts, and also from the experience

gained by the research group regarding the problems

https://musicascuola.indire.it/

https://lamusicaunisce.indire.it/

tinyurl.com/yddukmhp

https://www.gazzettaufﬁciale.it/eli/id/2017/05/16/

17G00068/sg

MusicBlocks: An Innovative Tool for Learning the Foundations of Music

477

that music teachers must face in the classroom with

respect to the teaching of certain concepts. With Mu-

sicBlocks, we tried to encourage the aspect of man-

ual skills even in a ﬁeld where the subject is actu-

ally purely abstract and not tangible, such as notes

and melody, where the main sense involved is gen-

erally not the touch but the hearing. Therefore, just

as a musical instrument needs hands (or other parts

of the body) to produce music, MusicBlocks was cre-

ated with the aim of making musical production tan-

gible, trying to manage the difﬁculties that arise from

learning to use a musical instrument. If this is not a

practical help in learning to play an instrument, it is

certainly helpful in acquiring the preparatory skills:

rhythm, melody, harmony, necessary to develop in-

terest in music. Starting from these premises, the re-

search was based on two different aspects. The ﬁrst

concerns the technological realisation of an educa-

tional tool for learning the fundamental principles of

music according to the goals, skill and competences

as described in (Miur, 2012, pp. 71-75). It combines

playful aspects (in particular the coloured bricks) with

educational aspects (in particular, in addition to the

musical aspects, problem solving and the learning by

doing approach). The other aspect concerns the peda-

gogical application that the use of this tool can have in

the classroom and the implications in terms of achiev-

ing speciﬁc competence goals and learning objectives

for primary and secondary schools.

3 MusicBlocks

MusicBlocks (Fig. 1) is a tool based on an open-

source architecture (Raspberry PI and software in

Python) that uses Computer Vision software to trans-

form the image of an alternative stave (acquired in

real time) into a melody.

Figure 1: The ﬁnal prototype of MusicBlocks.

This tool is designed to allow students to compose

their own melodies, to activate the competences ac-

cording to (Miur, 2012) and linked to the objectives to

be achieved in the ﬁrst cycle of education. The use of

MusicBlocks is very simple and intuitive: the bricks

represent the duration of the sounds, their position on

the plate represents the notes (pitch) and the play-

ing time. Once the students have placed the bricks

on the board, by pressing the Play button, the Mu-

sicBlocks will process the composition and play it as

a melody. This melody will be repeated cyclically un-

til the Stop button is pressed. MusicBlocks consists

of a base about 10 cm high that contains all the hard-

ware of the device, well hidden to ensure adequate

security. Housed on the base in full view and eas-

ily accessible, there is a removable plate which is the

work area. It is the alternative staff on which students

can work on creating their compositions. Finally, on

the edge near the plate, there are two buttons and a

LED light. Above the plate, at a height of about 30

cm, there are two cameras, mirroring the base and in-

clined by 30 degrees. Their function is to capture the

plate on which the students will place the Lego bricks

from two different angles, to obtain an adequate re-

dundancy of the data in order to overcome the prob-

lems caused by light gradients or reﬂections. On the

right side of the base are the connectors for recharging

the battery that powers the device and for connecting

it to the LAN network and the knob to increase or de-

crease the sound volume.

MusicBlocks is not just a physical device, it also

comes with a cloud platform on which every per-

formance is stored and easily accessible via a web

browser. The online environment, as well as acting as

a repository for the teacher, who can therefore control

the students’ work from the image of their composi-

tion to its sound performance, also has the character-

istic of reproducing the composition virtually, show-

ing the progress of the performance through a time-

line and also its transposition into traditional musical

notation.

3.1 MusicBlocks Features

For the realisation of the working area it was decided

to use a Lego plate of 16x16 pins. Imagining the

plate as a system of classical Cartesian axes, the y-

axis shows the notes (16 possible notes considering

the semi-tone) that vary gradually from C at the low-

est point of the diagram to D#, of the next octave, at

the highest point, depending on the height the brick is

placed, while the second (x-axis) represents the time

on which the notes unfold, expressed as 4 bars of 4/4

each (Fig. 2).

The bricks used in the musical composition repre-

sent a duration and are therefore of different lengths:

CSME 2022 - 3rd International Special Session on Computer Supported Music Education

478

Figure 2: Detail of the plate and musical notes in Italian

notation.

1/4, 2/4, 3/4 and 4/4. In addition, to make the recogni-

tion of their duration visually faster, it was decided to

use different coloured bricks for each type (Fig. 3).

Figure 3: Examples of bricks.

The arrangement of the bricks in the MusicBlocks

plate makes it possible to transform durations into

sounds. As soon as they are placed in relation to the

base representing time, at a speciﬁc pitch, they ac-

quire the meaning of musical notes. Once students

have completed the free arrangement of the bricks on

the plate, i.e.on the alternative stave, pressing the Play

button allows you to immediately listen to the perfor-

mance in a continuous loop. The Stop button, on the

other hand, interrupts music playback. MusicBlocks

is then automatically ready to handle changes to the

arrangement of the bricks and a new playback. As in-

dicated in Section 3 and in Section 3.3, a cloud en-

vironment has been created, accessible through the

WiFi network connection provided by MusicBlocks

itself (i.e. even without an active Internet connection),

in order to store every single compositional “perfor-

mance” and eventually to be able to:

• view an image of the plate and the bricks as it was

composed by students;

• listen to it again or download it in mp3 audio for-

mat;

• listen to the melody by displaying on the plate the

timeline of the notes played and at the same time

the transposition of the melody using traditional

musical notation (stave);

• rename the performance and change the musical

tempo (bpm) of the playback.

3.2 Design and Technological Solutions

The research activity relating to the technological de-

velopment of MusicBlocks involved two phases of

work. The ﬁrst concerned the construction of a pro-

totype device, i.e. the design of the object, which in-

volved various development phases aimed at optimis-

ing the ﬁnal result, and the choice of the most conve-

nient hardware solutions in terms of cost and perfor-

mance. The second phase concerned the development

of the software for transforming the physical com-

position into a melody, which also involved various

phases of reﬁnement in order to be able to model even

less efﬁcient lighting situations. In the development

of the software since we were working on images in-

tended as a matrix of pixels without any information

at the context level, we used advanced techniques of

image processing and Computer Vision as object de-

tection operations and image classiﬁcation.

3.2.1 Design Phase

The design of MusicBlocks and its realisation rep-

resented an important work of research and devel-

opment because there was no device similar neither

for functionality nor for architecture in literature and

in the market. Consequently, the physical design of

the instrument was a long process of realization and

revision, starting however from some considerations

shared by the research group on the characteristics

that it should have satisﬁed. In particular, the follow-

ing requirements were identiﬁed:

• ease of use;

• the reduction of the complexity for users;

• a wide and practical access to the area of manual

interactions;

• the use of standard bricks that do not require any

modiﬁcation with respect to those on the market;

• total safety in the use of the product;

• high ease of transport;

• the possibility of operating without any kind of

physical connection (i.e. without connection to

the electricity network or a LAN);

• open-source architecture and software;

• low implementation costs.

Ease of use is the requirement that has guided the

entire design in practice. In fact, the product is ex-

tremely simple and designed for minimal user interac-

tion: in addition to the positioning of the bricks, there

are only three buttons to interact with (power, play,

stop), while the technology is hidden from the user. In

order to meet the other criteria, MusicBlocks was de-

signed as a stand-alone product, i.e. autonomous and

without the need for additional third-party supports to

operate or special maintenance and/or precautions. In

the initial prototype, to make it easily transportable

and safe to use, a box-shaped structure with a lid was

MusicBlocks: An Innovative Tool for Learning the Foundations of Music

479

designed, later replaced by an open version without a

lid to better manage reﬂections due to external light-

ing. MusicBlocks was born as an open-source device

that could be easily replicated by anyone. With a com-

mon 3D printer, it is possible to print the various parts

that make up the external structure of the device: in

particular, the lower part, consisting of a hollow base

with accommodation for hardware and the Lego com-

position plate. In the lower part of the box are housed

(invisible to the user) all the hardware components

necessary for operation. In order to share the project

with schools and teachers, each component has been

chosen to be easily available on the market and at a

relatively low cost. As mentioned above, the design of

the MusicBlocks has undergone many changes over

time, as the experiments conducted on each sample

led to the identiﬁcation of some defects that could not

be overcome by software.

As shown in Fig. 4 we passed from a very ﬁrst

example of a stand-alone plate with camera support

(Fig. 5), to a closed version with lid and one camera

(Fig. 6), to a version with lid but two cameras, to the

ﬁnal version without lid and with two cameras per-

pendicular to the base and positioned in the middle

symmetrical to the plate (Fig. 1).

The ﬁnal prototype adopted in the ﬁrst phase of

experimentation, therefore, consists of a Raspberry Pi

3B+, an Arducam shield for the dual camera, 2 Raspi-

cams, an audio ampliﬁer, two mini speakers and a

power bank.

3.2.2 Hardware Equipment

Among the requirements identiﬁed in the early stages

of design, as indicated in Section 3.2.1, particular im-

portance was given to these three aspects: the selec-

tion of hardware components especially regarding to

the speed of execution, i.e. the time that elapses be-

tween when the play button is pressed and when the

melody is played; the ability to use MusicBlocks even

without connection the mains ; and the design in order

to have a device easily to be transported. Although the

possibility of using an Arduino-type electronic proto-

typing board as the core of the project was initially

considered, as it allowed for the management of a

number of external devices, it soon became clear that

in order to optimise the execution time of the image-

to-sound transformation software, i.e. the process-

ing time, higher performance in terms of computing

power and different hardware set-ups were required.

For this reason, the choice fell on SBC (Single Board

Computer) solutions, which have been on the market

for several years and have already been partly tested

in other Indire research projects. The board chosen

to implement the software and manage the hardware

components was the Raspberry Pi 3B+ because it has

a series of fundamental characteristics: ﬁrst of all

the possibility to be powered by a standard power

bank (like those for mobile phones), moreover with

1.4GHz frequency, 64 bit, 4 cores and 1GB of RAM it

guaranteed a computing power adequate to the needs

of the project, it had limited size and cost, a wide

range of connection types (WiFi, Bluetooth). Rasp-

berry has a very active online community and there-

fore has a wide range of software, libraries and docu-

mentation that have been extremely useful to support

the development of MusicBlocks facing problems in

the development. Starting from this board, therefore,

all the other needed components were identiﬁed: the

cameras are Raspicam v.2.1 (natively compatible with

the main board), a 20,000 mA power bank sufﬁcient

to power the circuit for many hours and an audio sys-

tem consisting of a small ampliﬁer and two speakers.

In addition to this, other electronic components were

used to build the interface, such as the two main but-

tons, the two LEDs, cables and resistors. Since the

Raspberry Pi board does not have a power button but

the system starts up when it receives power and the

shutdown phase has to be initiated by software inter-

face, the problem had to be addressed and in the last

prototype it was solved by software. While the de-

velopment of the ﬁrst prototypes, which included a

single camera, did not bring any particular compli-

cations at the hardware level, the development of the

last model with a double camera has involved a fur-

ther phase of study and experimentation because the

Raspberry Pi board provides for the management of

a single webcam through a hardware interface and, in

order to keep costs down and still ensure a good exe-

cution speed (which in the case of stereo image acqui-

sition involves a double processing), the possibility

of using USB webcams, which are generally slower,

more expensive and with lower deﬁnition levels, was

excluded. The problem was overcome by using the

Arducam shield, which, born in the context of video

control systems, is able to connect up to 4 Raspicam

directly to the only port available on the Raspberry Pi

board.

3.2.3 The Implementation Phase

The software of MusicBlocks can be divided into

three parts: the ﬁrst concerns the management of

hardware components (i.e power on, power off and

connection with the webcams), the second concerns

the processing of images and the encoding of the stave

in a MIDI ﬁle (Fig. 7 ), the third one manages the ex-

ecution of the MIDI ﬁle.

Leaving aside the algorithms for the management

of parts one and three, which do not present interest-

CSME 2022 - 3rd International Special Session on Computer Supported Music Education

480

Figure 4: Evolution of MusicBlocks design.

Figure 5: First example of MusicBlocks.

Figure 6: MusicBlocks with lid.

ing research elements, but are simply operational, let

us see in detail the functioning of the processing start-

ing from the acquisition of a left (ImL) and a right

(ImR) image of the plate, that are specular in posi-

tion and perspective distortions. Fig. 7 shows a high-

level but exhaustive diagram of the software blocks

and operations involved. Before proceeding to the ac-

tual processing, the images are binarized, processed

by means of the Canny Edge detector (Canny, 1986)

in order to identify the edge of the plate and cut out its

contour, and ﬁnally a rectiﬁcation of the images is ap-

plied to remove the perspective distortion introduced

by the lateral position of the webcam, by means of

homography. As it is evident from Fig. 8 and Fig. 9

it is possible to identify on each plate two grids that

are shifted between them but, given the ﬁxed position

of the plate, they are calculated a priori and stored as

initial parameters. The ﬁrst grid is calibrated with re-

spect to the plate and each position corresponds to the

central point of each pin. The second one is calibrated

with respect to the height of the brick. It is precisely

because these two matrices of points are identiﬁed a

priori and considered valid for each processing, that it

is very important that the cameras always acquire the

images from the same position, since a shift of even

a few pixels could invalidate the correct superimposi-

tion of the grids.

Using two images, each of which is analysed on

two levels, has produced beneﬁts in terms of accuracy

of results compared to previous prototypes where pro-

cessing was on a single image and single grid, which

repaid the greater processing time required. Follow-

ing the scheme of Fig. 7 before proceeding to the ac-

tual analysis of the plate, an algorithm was imple-

MusicBlocks: An Innovative Tool for Learning the Foundations of Music

481

Figure 7: MusicBlocks process.

Figure 8: Right Image: grid calibrated with respect to the

bricks (on the left) and to the plate (on the right).

Figure 9: Left Image: grid calibrated with respect to the

bricks (on the left) and to the plate (on the right).

mented for the calibration of the colours relative to

the bricks, mainly to take into account the lighting

context in which the MusicBlocks are positioned. In

the ﬁrst prototypes a static reference RGB colour ma-

trix was used, which was the result of an analysis of

about 100 colour samples in different lighting condi-

tions, but the results were not adequate. In order to

implement the dynamic colour matrix, ﬁve reference

bricks were inserted on the edges on the four sides of

the reading plate. The process then starts by taking

a small portion of the coloured bricks at pre-set posi-

tions from the two images in Fig. 10 on the right and

left.

For each image, each colour will have four dif-

ferent reference samples to compensate for any dif-

ferences in plate illumination: the average RGB of

each colour channel is then averaged. Each of the four

Figure 10: Samples of coloured bricks in right and left im-

ages.

reading areas undergoes a quality check before being

assessed as suitable for inclusion in this average. In

practice, to avoid cases in which there are reﬂections

and therefore areas that are not suitable for processing

(white pixels), the number of white pixels (accord-

ing to a pre-set threshold) is evaluated. If this num-

ber exceeds a certain threshold, the entire area is ex-

cluded from the calculation of the channel average. If

a colour has 2 or more areas excluded due to the pres-

ence of reﬂections, the processing will be interrupted,

and an error message produced. Once the RGB av-

erages of the reference colours have been calculated,

we move on to the actual image processing. Let us

consider ImR and ImL images and their grids at plate

and brick level (Fig. 8 and Fig. 9): for each of them

we have 256 areas of interest (portions of 3x3 pixels)

that must be analysed. The evaluation of the colours

associated with each portion is carried out ﬁrst glob-

ally for the right image and then for the left image,

ﬁrst for the grid at plate level and then for the grid at

brick level. Scrolling the sections from left to right

and from top to bottom, each element is subjected to

the same quality control as previously mentioned. If

the number of white pixels exceeds a certain thresh-

old, that section is discarded and the section in the

other grid is evaluated or otherwise only the section

in the other image is evaluated. For each suitable sec-

tion, the average colour is calculated and the distance

in three-dimensional Euclidean space from each of

the sample colours is determined. The section is then

assigned the colour with the smallest Euclidean dis-

tance from the samples. For each side (left and right),

CSME 2022 - 3rd International Special Session on Computer Supported Music Education

482

at the end of this process, each of the 256 sections of

the plate in the two images will be associated with a

colour: if the two grids identify the same colour, there

is no further work to be done, otherwise the one with

greater reliability is chosen.

Figure 11: On the left: identiﬁed colours of bricks at plate

level; on the right identiﬁed colours of bricks at pin level.

In Fig. 11 an example for the right images is

shown: in the ﬁrst row at plate level more bricks had

been identiﬁed with the right colour (“a” that is or-

ange) while at pin level some bricks had been iden-

tiﬁed as “x” which means “background”. The ﬁnal

conﬁguration of the bricks will then be obtained by a

merge between the colours identiﬁed on the right and

left side, together with considerations regarding the

number of consecutive coloured bricks, considering

that each type of brick has different lengths as well as

different colours.

Figure 12: Final conﬁguration.

Finally, once the ﬁnal conﬁguration has been de-

ﬁned as shown in Fig. 12, starting from the position

of the bricks and their colours, the MIDI ﬁle is recon-

structed through the MIDIUtil library and then sent

for execution by means of the PyGame library which

provides a player.

3.3 The Cloud

The functions of MusicBlocks are not limited to the

reading, analysis and audio reproduction of the com-

positions created. Parallel to this main activity, an

archive is populated in a completely autonomous way,

capable of storing all the compositions created. Ac-

cess to this archive is via an external device, such as

a computer, tablet or even a smartphone with a WiFi

connection. MusicBlocks is also a web server with

its own WiFi network that teachers and students can

connect to in order to perform certain actions. In this

mode, it is possible to review all the compositions cre-

ated, listen to them again and analyse them in more

detail. Each composition is displayed with an image

representing the position of the bricks. By clicking

on the desired image, it is possible to access speciﬁc

functions for in-depth analysis of the composition: in

addition to the image of the plate with the bricks,

there is also a stave (created using the open-source

JavaScript library VexTab) where the bricks are trans-

lated and transcribed into musical notation, and a

MIDI player that can play the composition again, also

being able to modify the BPM speed. While listen-

ing, the bricks that are producing the sound and the

relative note on the staff will be highlighted to im-

prove understanding of the sound-brick-notation as-

sociation. In this screen it is also possible to tag

the composition by entering useful information for

teachers such as the author, the date, the class that

can identify the type of exercise or other information

deemed important that allow them to create subsets of

compositions, related to a particular topic or exercise,

and make it easier to search for them later. Finally,

each composition includes buttons for downloading

the corresponding images or the generated MIDI ﬁle

to your device. From this panel you can also access

through an administrator user and thus have functions

dedicated to the maintenance of MusicBlocks such as,

for example, the possibility of updating the software

with the versions that will be released over time, or

share compositions considered particularly interest-

ing with the Indire repository, or receive assistance,

via log ﬁles, from the Indire project group, in case of

hardware malfunctions. To access these functionali-

ties there is a login that will allow the person in charge

of managing MusicBlocks (generally the teacher him-

self) exclusive access. The authentication login serves

both to ensure awareness of the execution of certain

procedures, but also to maintain a simple interface for

pupils.

MusicBlocks: An Innovative Tool for Learning the Foundations of Music

483

4 CONCLUSIONS

Learning how to play a musical instrument is not easy,

it requires great commitment and a good knowledge

of the theoretical foundations and fundamentals of

music. Currently, the “Italian National Indications”

propose activities and competence goals mainly ori-

ented towards secondary schools. MusicBlocks is an

instrument that can also be used from the ﬁrst cycle

of schooling and can help to bring students closer to

the concepts of rhythm, harmony and composition in

a playful and active way, overcoming the steep ini-

tial step that there is in learning to play an instru-

ment. MusicBlocks has been tested in a low sec-

ondary school and the results of this research will

be published shortly. Anyway, it is evident, from a

preliminary analysis, that the impact on learning and

teaching was signiﬁcant and the students’ satisfaction

very high.

REFERENCES

Addessi, A., Luzzi, C., Baroni, M., and Tafuri, J. (1996).

The development of music stylistic competence in

children. Bulletin of the Council for Research in Mu-

sic Education, 127:8–15.

Addessi, A. and Piras, E. (2020). L’educazione musi-

cale negli anni ottanta, un’epoca di cambiamenti,

bibliomanie. Letterature, storiograﬁe, semiotiche,

(50):1–9.

Ausubel, D. P. (1968). Educational psychology: a cognitive

view. Educational psychology: a cognitive view. Holt,

Rinehart and Winston, New York, NY, USA.

Barat

e, A., Ludovico, L., and Malchiodi, D. (2017). Foster-

ing computational thinking in primary school through

a lego

-based music notation,. Procedia Computer

Science, vol. 112:1334–1344.

Baroni, M. (1978). Suoni e Signiﬁcati. Musica e attivit

espressive nella scuola. Edt, Torino.

orner, H. (2007). Functional polymer-bioconjugates as

molecular lego

 bricks. Macromol. Chem. Phys.,,

208:124–130.

Bottero, E. and Carbone, I. (2003). Musica e creativit

a – La

didattica di Giordano Bianchi. Milano.

Cairo, M. T. (2012). Gioco, attivit

a ludica e

costruzione dell’identit

a nel bambino:l’importanza

dell’apprendimento signiﬁcativo,. In Cairo, M. T., ed-

itor, Percezione, corporeit

a e apprendimento in didat-

tica e in riabilitazione, pages 189– 228. Vita e Pen-

siero, Milano.

Campbell, D., Miller, D., Bannon, S., and Obermaie, L.

(2011). An exploration of the nanoworld with lego

bricks. Journal of Chemical Education, 88(5):602–

606.

Caon, F. and Rutka, S. (2004). La lingua in gioco. Perugia.

Capuano, A., Storace, F., and Ventriglia, L. (2018).

Apprendimento signiﬁcativo. Utilizzo didattico delle

mappe concettuali. Lattes, Torino.

Celli, P. and Gonella, S. (2015). Manipulating waves with

lego

 bricks: A versatile experimental platform for

metamaterial architectures. Applied Physics Letters,

107(8):081901.

Delfrati, C. (1979). Progetti sonori. Napoli.

Della Casa, M. (1985). Educazione musicale e curricolo.

Zanichelli,, Bologna.

Jonassen, D. H., Howland, J., Marra, R. M., and Crismond,

D. P. (2007). Meaningful Learning with Technology.

Pearson, Upper Saddle River, NJ., 3

◦

ed. edition.

Krashen, S. D. (1983). Principles and Practice in Second

Language Acquisition. Pergamon Press, Oxford.

Ludovico, L. A., Malchiodi, D., and Zecca, L. (2017).

A multimodal lego

-based learning activity mixing

musical notation and computer programming. In Pro-

ceedings of the 1st ACM SIGCHI International Work-

shop on Multimodal Interaction for Education, MIE

2017, pages 44–48, New York, NY, USA. Association

for Computing Machinery.

Mattheij, J. (2022). Personal web page.

Merlo, D. (2017). La robotica educativa nella scuola pri-

maria,. Teli, ebook edition.

Miur (2012). Indicazioni nazionali per il curricolo della

scuola dell’infanzia e del primo ciclo d’istruzione. In

Annali della Pubblica Istruzione, volume lxxxviii.

Oestermeier, U., Mock, P., Edelmann, J., and Gerjets, P.

(2015). Lego music: Learning composition with

bricks. In Proceedings of the 14th International Con-

ference on Interaction Design and Children, IDC ’15,

pages 283–286, New York, NY, USA. Association for

Computing Machinery.

Papert, S. (1984). Mindstorms. Bambini, computer e cre-

ativit

a. Emme, Milano, 1980 original edition.

Piazza, G. (1979). Orff-Schulwerk. Musica per Bambini.

Manuale. Edizioni Suvini Zerboni, Milano.

Piazza, G. (1984). Orff-Schulwerk. Musica per Bambini.

Esercitazioni pratiche. Edizioni Suvini Zerboni, Mi-

lano.

Porena, B. (1979). Musica prima. La composizione mu-

sicale. Uno strumento della pratica culturale di base

nella scuola e nel territorio,. Albarea, Treviso.

Rizzo, A. L. (2014). Didattica ludica e giochi

musicali nella prospettiva inclusiva: il ruolo

dell’insegnante/musicista di sostegno.

Rogers, C. (1973). Libert

a nell’apprendimento. Giunti Bar-

bera, Firenze.

Salmaso, L. (2013). Le potenzialit

a del gioco con i mat-

toncini lego nella scuola primaria. TD Tecnologie Di-

dattiche, 21(3):168–174.

Salmaso, L. (2014). Costruisco e imparo. Giochi e attivit

con i mattoncini LEGO

 per lo sviluppo di abilit

visuo-spaziali e di pianiﬁcazione. Erickson.

Stefani, G. (1982). La competenza musicale. CLUEB,

Bologna.

Sudati, I. (2014). La didattica ludica. teoria e applicazioni

pratiche nell’insegnamento dell’italiano l2 ad adulti.

Italiano LinguaDue, 5.

Valentini, P. and Tallandini, M. (1998). Gli stadi dello

sviluppo. In SEMPIO, O. L., editor, Vygotskij, Pi-

aget, Bruner. Concezioni dello sviluppo, pages 160–

163. Raffaello Cortina.

CSME 2022 - 3rd International Special Session on Computer Supported Music Education

484