Improvement Model of Lecturer Ability in Preparation of Assessment

Instruments Civic Education in Higher Education

Wijianto, Winarno, Rini Triastuti

Faculty of Teacher Training and Education, Sebelas Maret University, Surakarta, Indonesia.

Keywords: Model; Assessment Instruments; Civic Education; Higher Education

Abstract: This study aims to produce a model for improving the ability of lecturers in the preparation of assessment

instruments for civic education in universities. The method used in the form of ADDIE instructional design

development model with stages of Analysis, Design, Development, Implementation, Evaluation. Resarch

involves lecturers of civic education in the city of Surakarta. The results showed that the model of

improving the ability of lecturers in the preparation of assessment instruments for civic education in higher

education at the stage of needs analysis found problems, not reflecting the characteristics of students, not

oriented to the assessment component of civic education, there is no instrument for assessment of civic

dispotision and civic skill.

1 INTRODUCTION

In Learning analytics can improve learning practice

by transforming the ways we support learning

processes (Olga, Mathias, Olof, Anna, 2018). The

learning process in higher school or universities is

highly expected to use innovative learning models,

which did not develop cognitive aspects only, but

can achieve affective and psychomotor aspects. The

implementation of Civic Education in universities is

realized in the form of learning, which are includes

planning, implementing, and evaluating. These three

stages are designed towards that they cannot be

separated from each other to achieve a learning goal

(Liu, Kang, Zou, Lee, Pan, Corliss, 2017)

The planning stage is the initial stage in learning

activities that are very important as signs in the

implementation of learning itself. In-depth analysis

is needed to design learning plans. So that learning

planning is obtained which accommodates the three

expected competencies, namely cognitive, affective,

and psychomotor aspects (Jivet, Scheffel, Specht,

Drachsler, 2018)

The education paradigm is related to 4 (four)

things that are the basis for the implementation of

education, namely students, lecturers, materials and

education management. In implementing education,

there are at least two poles of paradoxical paradigm,

namely the feudalistic paradigm and the humanistic

paradigm. The feudalistic paradigm has the

assumption that educational institutions

(universities) are a place to train and prepare

students for the future. (Rienties and Toetenel, 2016)

Therefore, students are placed as objects only in

learning, while lecturers as the only source of

knowledge of truth and information, behave

authoritarian and bureaucratic. Learning material is

structured rigidly so that it encapsulates the

creativity of students and lecturers. (Rubel and

Jones, 2016) Meanwhile, education management

including learning management is centralistic,

bureaucratic and monolithic. In applying the

learning strategy, it is very dogmatic, indoctrinative

and authoritarian.

Meanwhile the humanistic paradigm based on

the assumption that students are human beings who

have different potential characteristics. Therefore, in

this view students are placed as subjects as well as

objects of learning, while lecturers are positioned as

facilitators and student dialogue partners.

(Brickhouse, 1990) Learning materials that are

prepared based on the basic needs of students, are

flexible, dynamic and phenomenological so that the

material is contextual and has relevance to social

demands and changes. Also the management of

education and learning emphasizes on a

decentralized, non-bureaucratic dimension,

recognizing plurality with the use of varied and

democratic learning strategies. Observing the

Wijianto, ., Winarno, . and Triastuti, R.

Improvement Model of Lecturer Ability in Preparation of Assessment Instruments Civic Education in Higher Education.

DOI: 10.5220/0009024100002297

In Proceedings of the Borneo International Conference on Education and Social Sciences (BICESS 2018), pages 575-579

ISBN: 978-989-758-470-1

575

direction of change and refinement of the signs for

the implementation of the Civic Education Course

that has been determined by the Directorate General

of Higher Education, has indicated using the

humanistic paradigm. (Hancock and Gallard, 2004)

However, the learning process is closely related

to the evaluation system. To be able to know the

success of a learning process, the evaluation system

prepared must be in accordance with the paradigm

and characteristics of learning civic education.

Therefore, the planning of assessment instruments is

very important and must be truly mastered by the

lecturer in preparing the instructional design course.

This article discusses the data of model for

improving the ability of lecturers in the preparation

of instruments for assessing civic education in

universities..

2 LITERATUR REVIEW

There are several stages of the design process which

are divided into four activities, namely: constructing

criterion referenced test, media selection, selection

format, initial design. Activities carried out at this

stage include: 1) Compile the criteria test, as the first

action to find out the initial ability of students, and

as an evaluation tool after the implementation of the

activity. 2) Selecting learning media that is in

accordance with the material and characteristics of

students. 3) Selection of the form of presentation of

learning adapted to the learning media used. If the

teacher will use audio visual media, at the time of

learning, of course students are told to see and

appreciate the audio visual media show. 4) Simulate

the presentation of material with the media and the

learning steps that have been designed. When the

learning simulation takes place, assessment is also

carried out by peers (Oster, Lonn, Pistilli, and

Brown, 2016)

In the design stage, researchers have made a

prototype or product design. In the context of the

development of teaching materials, this stage is

carried out to make modules or textbooks in

accordance with the content framework of the results

of curriculum and material analysis (Nistor, Demtl,

and Klamma, 2015) In the context of developing

learning models, this stage is filled with activities to

prepare the conceptual framework of learning

models and tools (material, media, evaluation tools)

and simulate the use of the learning models and

devices in a small scope (Banta and Associates,

2002)

Before the product design continues to the next

stage, the product design (models, textbooks, etc.)

needs to be validated. Product design validation is

carried out by peers such as lecturers or teachers

from the same field of study/expertise (Rienties,

Toetenel, Bryan, 2015) Based on the results of the

peer validation, there is a possibility that the product

design still needs to be corrected according to the

validator's advice.

Another importent process within instructional

design is a division of the development stage which

is devided in two activities, namely: expert appraisal

and developmental testing (Wise, Vytasek,

Hausknecht, and Zhao, 2016) Expert appraisal is a

technique to validate or assess the feasibility of

product design. In this activity an evaluation was

conducted by experts in their fields. The suggestions

given are used to improve the material and learning

designs that have been compiled. Developmental

testing is the activity of testing product designs on

the real target subject. At the time of this trial data

was searched response, reaction or comment from

the target user of the model. Test results are used to

improve the product. After the product has been

repaired, it is re-tested until it has effective results

(Williams, 2017)

In the context of instrument development, the

development stage is carried out by analyzing

instrument requirements and then testing the

instrument products to the experts involved during

design validation. The test results are then used for

revision so that the instrument really meets the needs

of users.

3 METHODS

This research was conducted at the Sebelas Maret

University, Surakarta, in the general subject of

Citizenship Education. The object of the research

study is the design instructional learning plan made

by all Lecturers of the General Lecture on Civic

Education at the Sebelas Maret University,

Surakarta. To construct a learning assessment design

model, researchers dig up information through Focus

Group Discussion with experts who have expertise

and experience within Civic Education Subject at

Sebelas Maret University, Surakarta.

This research used ADDIE instructional design

development model. ADDIE was Analysis, Design,

Development or Production, Implementation or

Delivery and Evaluations. ADDIE model developed

by Dick and Carry (1996) an explanation of ADDIE,

as follows:

BICESS 2018 - Borneo International Conference On Education And Social

576

3.1 Analysis

At this stage, the main activity is to analyze the need

to develop new learning models/methods and

analyze feasibility and the requirements for

developing new learning models / methods. The

development of new learning methods begins with

an internal problem learning model / method that has

been applied.

3.2 Design

In designing learning models/methods, the design

phase has similarities with designing teaching and

learning activities. This activity is a systematic

process that starts from setting learning goals,

designing scenarios or teaching and learning

activities, designing devices learning, designing

learning materials and learning outcome evaluation

tools. The design of the model/learning method is

still conceptual and will underlying the next

development process.

3.3 Development

Development in the ADDIE model contains design

realization activities product. In the design stage, a

conceptual framework for implementation has been

prepared new learning model/method. In the

development stage, the framework it is still

conceptualized into a ready product implemented.

For example, if the design stage has been designed

use of new conceptual models/methods, then at the

stage development is prepared or made with learning

tools new models/methods such as lesson plans,

media and subject matter.

3.4 Implementation

At this stage the design and methods have been

implemented developed in a real situation that is in

class. During implementation, the design of the

model/method that has been developed is applied to

conditions which are actually. The material is

delivered according to the new model/method

developed. After applying the method then an initial

evaluation is carried out to provide feedback on the

application of the next model/method.

3.5 Evaluation

Evaluation is carried out in two forms, namely

formative and summative evaluation. Formative

evaluation is carried out at the end of each meeting

(weekly) while summative evaluation is carried out

after the activity ends in its entirety (semester).

Sumative evaluation measures the final competency

of subjects or learning objectives to be achieved.

Evaluation results are used to provide feedback to

parties user model/method. Revisions are made in

accordance with the evaluation results or needs that

have not been met by the new model/method. The

table bellow showed ADDIE process in this study:

Table 1. ADDIE Development Design

The Procces of ADDIE in Improvement Model

A Analysis

Analyze the lack of instructional

design courses for civic education

in universities

D Design

Make a mapping design of

instructional design courses for

civic education in universities

D Develop

Develop a model of instructional

design courses for civic education

in universities

Implemen

Implementing a model of

instructional design courses for

civic education in universities

E Evaluate

Evaluating the model of

instructional design courses for

civic education in universities

4 RESULT AND DISCUSSION

Based on document review and focus group

discussion, researchers found that the majority of

instructional design for civic education in higher

education made by lecturers was oriented to the

cognitive/knowledge aspect, which reached 80% of

the total 92 Classes. Of the total 80% of cognitive-

oriented instructional design, 60% have low

cognitive levels. This data showed that the thinking

orientation of the lecturers of civic education is

dominated by cognitive orientation, thus the

assessment process will only lead to cognitive

evaluation only.

As for more detailed data regarding instructional

design courses on civic education are as follows:

Improvement Model of Lecturer Ability in Preparation of Assessment Instruments Civic Education in Higher Education

577

Figure 1. Instructional Design Problem Civic Education

in Universities

Based on the chart above, we know that

instrukctional design problem within civic education

learning process in higher education are 82 or about

90% of the 92 instructional designs studied were

found not to be oriented towards Civic Disposition

and Civic Skill. 69 instructional designs (75%) out

of a total of 92 instructional designs found errors in

the use of operational verbs in the learning process.

72 or about 78% of the 92 instructional designs

studied, found that there were no measurable ability

stages so that the target of learning achievement was

not known and what. 78 or about 85% of the 92

instructional designs studied, are not known or not

included about the formulation of attitudes in the

learning process.

Other data found related to assessment

instruments on civic education instructional design

are as follows:

Figure 2. The Problem of Affective and Skill's Assessment

There are problems in the affective assessment

planning process and skills in instructional design.

90% or around 83 of the total 92 RPS studied, there

were no affective and skill assessments. That is, only

10% of the overall instructional design that has the

completeness of the assessment process planning is

cognitive assessment, affective assessment, and

psychomotor assessment.

The problem in the instructional design planning

process carried out by lecturers in civic education

courses in universities is dominated by immature

planning problems at the planning stage of

assessment instruments which are still difficult in

compiling their assessment instruments. Lecturers

find it difficult to develop operational verbs in the

affective and skill assessment stages. As for the

cognitive assessment stage, lecturers of citizenship

education did not experience problems.

Based on the data, researchers have made an

improvement model of lecturer ability in preparation

of assessment instruments civic education in higher

education bellows:

Figure 3. Improvement Model in Preparation of

assessment instrument

5 CONCLUSIONS

The results showed that the model of improving the

ability of lecturers in the preparation of assessment

instruments for civic education in higher education

at the stage of needs analysis found problems, not

reflecting the characteristics of students, not oriented

to the assessment component of civic education,

there is no instrument for assessment of civic

dispotision and civic skill. Lecturers find it difficult

to develop operational verbs in the affective and

skill assessment stages. As for the cognitive

assessment stage, lecturers of citizenship education

did not experience problems.

REFERENCES

Banta, T. W. & Associates. Building a Scholarship of

Assessment. San Francisco: Jossey-Bass, 2002. Essays

by leaders in the field, addressing practical issues, but

focusing on developing a “scholarship of assessment.”

Bibliography provides recent references to more

specialized works on designing and selecting

assessment instruments and other topics.

InstructionalDesignProblem

Orientation

assessment

instrument

abilitystage

disposition

assessment

10%

90%

Complete

assessment

Inomplete

assessment

Phase1

AnalysistheProblem

Phase2

MappingStudents

Characteristic

Phase3

EvaluatingAssessment

Component

BICESS 2018 - Borneo International Conference On Education And Social

578

Brickhouse NW. 1990. Teachers’ beliefs about the nature

of science and their relationship to classroom practice.

J Teach Educ 3:53–62

Dick, W and Carey, J. O. 1996. The Systematic Design of

Instruction. New York,Longman.

Hancock ES, Gallard AJ. 2004. Preservice science

teachers’ beliefs about teaching and learning: the

influence of K-12 field experiences. J Sci Teach Educ

15:281–291

Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. 2018.

License to evaluate: Preparing learning analytics

dashboards for educational practice. Proceedings of

the 8th international conference on learning analytics

& knowledge (pp. 31–40). ACM.

Liu, M., Kang, J., Zou, W., Lee, H., Pan, Z., & Corliss, S.

2017. Using data to understand how to better design

adaptive learning. Technology, Knowledge and

Learning, 22(3), 271–298.

Nistor, N., Derntl, M., & Klamma, R. 2015. Learning

Analytics: Trends and issues of the empirical research

of the years 2011-2014. In G. Conole et al (Ed.).

Design for teaching and learning in a networked

world: Ec-tel 2015, LNCS 9307 (pp. 453–459).

Springer.

Olga Viberga,∗, Mathias Hatakkab, Olof Bältera, Anna

Mavroudia. 2018. The current landscape of learning

analytics in higher education. Computers in Human

Behavior 89, pp. 98–110.

Oster, M., Lonn, S., Pistilli, M. D., & Brown, M. G.

(2016, April). The learning analytics readiness

instrument. Proceedings of the sixth international

conference on learning analytics & knowledge (pp.

173–182). ACM.

Rienties, B., & Toetenel, L. 2016. The impact of 151

learning designs on student satisfaction and

performance: Social learning (analytics) matters.

Proceedings of the sixth international conference on

learning analytics & knowledge (pp. 339–343). ACM.

Rienties, B., Toetenel, L., & Bryan, A. 2015. “Scaling up”

learning design: Impact of learning design activities on

LMS behavior and performance. Proceedings of the

fifth international conference on learning analytics and

knowledge (pp. 315–319). ACM.

Rubel, A., & Jones, K. 2016. Student privacy in learning

analytics: An information ethics perspective. The

Information Society, 32(2), 143–159.

Williams, P. 2017. Assessing collaborative learning: Big

data, analytics and university futures. Assessment &

Evaluation in Higher Education, 42(6), 978–989.

Wise, A., Vytasek, J., Hausknecht, S., & Zhao, Y. 2016.

Developing learning analytics design knowledge in the

“middle space”: The student tuning model and align

design framework for learning analytics work. Online

Learning, 20(2), 155–182.

Improvement Model of Lecturer Ability in Preparation of Assessment Instruments Civic Education in Higher Education

579