Can Digital Footprints Save the Physical Lecture?

Tarjei Heggernes and Ole Jakob Bergfjord

Department of Business Administration, Western Norway University of Applied Sciences, Bergen, Norway

Keywords: E-learning, Digital Footprints.

Abstract: We argue that better use of “digital footprints” (data generated from students’ learning activities) could be

used to improve the traditional lecture. We point out some potentially important data sources, and briefly

discuss how data from each of these sources can contribute to better learning. Finally, we argue that even if

this development is possible, it will require changes – certainly changed priorities among faculty, but also

probably recruitment of new types of expertise, for instance data scientists.

1 INTRODUCTION

The death of the traditional lecture has been

announced at least since the Dearing Report was

published more than 20 years ago. (Dearing,

1997). As the longest standing form of teaching, the

lecture gets a lot of critic for being antiquated, partly

because we still follow patterns from lectures from

ancient times and ancient technologies, and partly

because it is more about teaching than learning.

However, why should lecturers change a winning

formula?

We know by now that every student learns in a

different way. Still, especially in higher education, we

have standardized courses for big classes. For us as

lecturers, high attendance is a measure of a good

course. Everybody knows that the best lecturer gets

the highest attendance, and praise from both students,

colleagues and other staff. But is this really the best

measure of whether a class is useful?

In a previous project (Bergfjord and Heggernes,

2016) we found that the use of “flipped classroom”

methodology led to higher grades for the students.

But for which students? Apparently, not the students

attending class, but the students for which the class

environment did not represent the best learning arena!

We saw no difference in the higher grades, the bigger

effect were on the lower grades. Somehow, by using

videos, the learning outside the classroom improved.

Also, in two informal surveys we recently conducted

on which methods/tools the students found most

useful in a course, the recorded lecture got the highest

score, the students ranked (the availability?) of the

recorded lecture more useful than attending lectures

in person.

Each year when we start a new class, we have a

brief discussion of the lecture as “quality time”.

Wikipedia refers to quality time as time spent with

partners, friends or family that is in some way

important, special, productive, or profitable. The

discussion usually ends with an emphasis on

productivity in reaching the goals of attending class.

The goals for the students by attending class could be

discussed at length, but a suggested goal is achieving

the highest possible grade by a minimum of effort.

Some students usually nod affirmatively. As a result,

there is more focus on assignments and less on

lecturing during the lectures, even for big classes.

We believe measuring and evaluating the activity

both in and outside class could be improved in most

traditional courses. Grades are still often given based

on one or a few tests or term papers, usually at the end

of the semester. This is comparable to a retail store

that only recorded total sales for each product every 6

months and used those numbers to analyse their

business. Furthermore, student evaluation of classes

and teachers are collected mostly in the same manner.

Research shows that there are lot of biases that

influence the student’s evaluation (see e.g. Seiler,

1999), and for voluntary evaluations, the response

rates are often low.

The background for this position paper is hence

the following: The lecture remains widely used,

although much evidence suggests it is often not the

best learning method. We assume the continued usage

somehow implies that the lecture has some qualities,

at least related to convenience and “bang for the

Heggernes, T. and Bergfjord, O.

Can Digital Footprints Save the Physical Lecture?.

DOI: 10.5220/0007762504610464

In Proceedings of the 11th International Conference on Computer Supported Education (CSEDU 2019), pages 461-464

ISBN: 978-989-758-367-4

461

buck”. On the other hand, new technology gives us

better opportunities to utilize data to measure and

improve all types of learning, including learning from

lectures.

Hence, our position is that better use of learning

intelligence will give better insight into how to

improve lectures, which in turn could save the

traditional lecture.

Learning intelligence is a relatively new academic

field. We are lecturers of business topics, and for most

business topics, there is a lot of emphasis on

collecting and using data as a basis for decisions. Yet,

as stated above, we do not collect much data from our

own classes as a basis for decisions concerning our

pedagogics. Learning intelligence in higher education

is still in its infancy (Viberg et al., 2018). There are

several definitions of learning intelligence, but we

will base our discussion on the following: “learning

analytics is the measurement, collection, analysis and

reporting of data about learners and their contexts,

for purposes of understanding and optimising

learning and the environments in which it

occurs” (Siemens and Long, 2011).

If we take a step back to briefly visit the business

world again, the earliest definition of a business

intelligence (BI) system dates back to 1958 (Luhn,

1958). The BI system is defined as a system that “will

utilize data-processing machines for auto-abstracting

and auto-encoding of documents and for creating

interest profiles for each of the "action points" in an

organization”. Is auto-abstracting and auto-encoding

even possible in an educational environment? As for

the action points within an organization, we clearly

see that lecturers, researchers, department heads,

administrative staff and students would all be

different profiles with different action/decision

points.

A few words about data quality: It is well-known

fact that even the best system relies on good data in

order to produce good data out. Good intelligence

requires good data. An important aspect of data

quality is data granularity (Kimball and Ross, 2011).

The granularity of data represents the level of detail

in the data and is tied to how often data is collected.

According to Kimball & Ross, good analysis requires

that the most detailed grain of information possible is

captured in computer systems. As seen over, the

detail of both grades and evaluations is very coarse.

Is it at all possible to get more specific information on

learning activities?

2 PROPOSED SOURCES OF

DATA

Every use of a digital resource from the students will

leave a digital trail, either anonymous or traceable.

More digital resources inevitably lead to more data.

Below, we discuss potential data-points and their

usage.

Sources of data mainly outside the classroom:

1. The Learning Management System (LMS)

At our institution, we use the Canvas LMS. Canvas

has lots of functionality for setting up quizzes and

assignments, learning activities that can produce good

and relevant data. There is also a function called

Course Statistics, where it is possible to view each

student’s activity on the platform. The detail of the

information could have been finer (but there is a hack

for obtaining more detailed data on the interactions

between the students and the course material). For

example, information on how many pages the student

has accessed, and when, is available, but not which

pages. On a course-level, frequently visited pages on

the LMS can help identify difficult topics, and lead to

a decision to repeat or present the topic a different

way in lectures.

2. Other Platforms for Digital Learning

Resources.

Video instructions are an important part of a flipped

classroom approach to teaching. Our experience is

that the most simple, convenient and effective

platform for distributing videos is YouTube. There

are good analytical tools available on the platform

that are easy to use. A premise

for the flipped classroom approach is that the

students watch the videos before the lecture. The

viewing numbers are live on YouTube, and easy to

check before or during a lecture. Low viewing

numbers will call for an explanation of the concepts

or theories in class before starting on assignments,

thus improving the quality of the lecture time.

In addition, the percentage of the video viewed,

e.g., when in the video the students stopped watching

the video could be valuable input, both for the

physical lectures and when re-recording the videos

later. Another similar platform is

Soundcloud, which can be used to publish

audio/podcasts. Listening statistics are available for

each episode and can be utilized much the same way.

CSEDU 2019 - 11th International Conference on Computer Supported Education

462

Sources of data within the classroom:

We think that all lecturers live for the discussions they

have with students in class, we certainly do. As good

as they can be, there are two potential problems with

class discussions. First, often only a small part

of the class will participate, and often the same

students each time. This can lead to a somewhat

limited number of perspectives in the discussion.

Second, even if the discussions are documented in a

lecture recording, the students participating are not

wearing microphones, so viewers of the recording

will not be able to hear the student part of the

discussion. In lectures that are not recorded, the

discussions will be unavailable for those students not

attending the physical lecture, and at best very hard to

recall for those attending and participating. (We

suspect that most of these will have forgotten the

whole discussion within days.)

If there were a way to structure and document

those discussions, there would be two obvious

benefits. For the student, the documentation would be

useful for repetition. For the lecturer, the

documentation would be available, and useful, when

preparing for the next time doing the same course. It

would be easier to assess if the questions were any

good for making the intended point during the lecture

and improving the structure of the lecture if

necessary. We propose the following ways to gather

data within the classroom:

1. Working on Assignments on a Shared

Document

We have several years' experience using MS

OneNote for this purpose. The platform is in no way

perfect or without its technical flaws; in fact we have

considered changing it several times. Still it is good

enough to both document the discussions and capture

more perspectives from the students. It is easy to

access without logging into a service, but as many

institutions now have Office365 for their students and

staff, logging in is also an option that will make it

easier to trace each student’s activity.

Practically, students are given an assignment for

discussion during the lecture, and a timeframe for

solving and writing down the answer, usually 5 or 10

minutes. In OneNote they will create a tab (in

OneNote this is called a page) with their names as

a heading and write down the answer on the page.

This answer will be available for all the other

students, and the lecturer, to see. The answers then

make the basis for the discussion, making it possible

for the lecturer to reach out to specific students by

name to start the discussion.

This way of data collection has proven to

constitute a valuable archive for both lecturer and

students.

2. Micro-surveys at the End of Each Lecture

Overwhelmingly long, non-compulsory evaluations

are not very appealing to anyone. This also applies to

students, and as mentioned before there are a lot of

sources of bias and error that affect the outcome of

those evaluations. The timing is one of them, right

before exams is a stressful time for

students that might affect the evaluation, and after the

exam, the grading can affect the evaluations both

positively and negatively. It is also hard to remember

specific classes and topics, so what is measured is

more a general impression. Thus, end-of-semester

evaluation results might not be the best data for

optimizing the learning and the environment in which

it occurs. At best, the next class will reap the benefits,

not the class doing the evaluations. We propose the

following:

We designed a framework for micro-surveys to serve

two purposes. First, to give the students a couple of

minutes for reflection at the end of the lecture.

Second, and most importantly for learning analytics,

give the students a chance to evaluate the lecture in

two quick ways: A 1 - 5 -star rating on how much

understanding they have for central topics presented

during the lecture, usually 4 or 5 topics, and a text

field where they can write down what is most unclear

to them after the lecture. In addition to this there is

also a field where the students can give general

comments to the lecture.

Many times, one or two of the topics will get a

lower rating than the others. Repetition of these

topics will be a good starting point for the next

lecture, and the same goes for discussing some

answers from the text field of unclear topics after the

lecture. The answers in the text field will help

uncover other topics for repetition than the pre-

defined topics that are star-rated.

Practically, this short survey is done with Forms

in Office365, and the students are given a link to the

form in the last picture for the lecture notes, where it

is also possible to access the survey on a mobile

device via a QR-code. The results from the previous

lecture will be presented in the beginning of the

lecture notes for the next lecture. Again, the data

collected makes it possible to improve the following

Can Digital Footprints Save the Physical Lecture?

463

lecture(s) for the same students, as well as generally

improving the course in following years.

3 THE NEED FOR A NEW

SKILL-SET

It is of course possible to start with small sets of data

for doing learning analytics. But data is getting

bigger, and with bigger data sets comes the need for

more advanced analytical tools. Using a hack, we

were able to retrieve the total interactions between the

students and learning material (on the LMS Canvas)

in a course. The course had 100 students, the activity

report was a .csv-file with over 8500 rows. A pivot-

table in Excel, or a more powerful tool like Power BI

will get a lecturer a long way in analysing these data,

but there are also, as we have suggested, more data to

be analysed. The amount of data will quickly become

overwhelming, which will pose a barrier for using

learning analytics in the first place.

Lecturers are pressed for time, there are high and

growing demands on performance and

documentation of teaching and research. For many,

the time for analysing large amounts of educational

data is just not there. Still, we think it is an important

task.

For quick and easy data browsing, like using a

shared document for assignment during class or a

micro-survey at the end of class, the regular lecturer

will, in our opinion be fine. For more thorough

analysis, a data scientist will be in order. Far from all

educational institutions see themselves as competing

in a data-driven market, but the truth is that especially

in higher education, they are. Competitors/substitutes

to higher education like EdX and Coursera are

basically in the technology sector and are analysing

data on how students learn online. The next step for

offline institutions will be hiring educational data

scientists. That might not be an easy task: Data

Scientists are in strong demand, and a data scientist

with domain knowledge in education might be as rare

as a unicorn.

4 CONCLUSIONS

The traditional lecture is widely used, despite its

obvious weaknesses. This is likely to continue, as

both teachers and students are conservative, and

many consider the lecture to be a convenient

alternative. On the other hand, there are large

amounts of data available, which only to a limited

degree are utilized. We discuss a few such data

sources and show how better use of these can improve

the traditional lecture, and thus contribute to its

survival. At the same time, we argue that this will

require a change of priorities for many faculty

members, and maybe also for universities to start

recruiting new types of employees, for instance data

scientists specializing in gathering and analysing

educational data.

REFERENCES

Bergfjord, O.J. & Heggernes, T. (2016) Evaluation of a

“Flipped Classroom” Approach in Management

Education. Journal of University Teaching & Learning

Practice, 13(5), 1-11.

Dearing, R. (1997) The Dearing report: Higher education in

the learning society. Retrieved from

http://www.educationengland.org.uk/documents/deari

ng1997/dearing1997.html

Kimball, R., & Ross, M. (2011) The data warehouse

toolkit: the complete guide to dimensional modeling.

John Wiley & Sons.

Luhn, H. P. (1958) A business intelligence system. IBM

Journal of research and development, 2(4), 314-319.

Seiler, M. J., Seiler, V. L., Chiang, D. (1999) Professor,

student, and course attributes that contribute to

successful teaching evaluations. Financial Practice

and Education, 9(2), 91-99.

Siemens, G., & Long, P. (2011). Penetrating the fog:

Analytics in learning and education. EDUCAUSE

review, 46(5), 30.

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018)

The current landscape of learning analytics in higher

education. Computers in human behaviour, 89, 98-

110.

CSEDU 2019 - 11th International Conference on Computer Supported Education

464