The Readability Level of English Textbook Published by

Kemendikbud for Grade Xi

Mustika Permatasari, Tati Lathipatud Durriyah, Ismalianing Eviyuliwati, Maya Defianty,

Neneng Sunengsih, Nasrun Mahmud and Wahyunengsih

Department of English Education, Faculty of Educational Sciences,

Universitas Islam Negeri (UIN) Syarif Hidayatullah Jakarta, Indonesia

Keywords: Flesch Reading Ease, Readability Level, Reading Texts.

Abstract: This study aims to examine and describe the readability level of reading texts in English textbook published

by the Ministry of Education and Culture (Kemendikbud) for the eleventh grade of Senior High School with

a descriptive analysis method. There were 11 reading texts of the English Textbook 2017 revised edition as

the object of this research. The instrument of this study was to collect data procedures and then measure with

the Flesch Reading Ease (FRE) formula. There were only two of the reading texts were found as the

appropriate level for eleventh-grade students. The data showed that only 18.18% of the eleven reading texts

were suitable for eleventh-grade students. The suitable level was Fairly Difficult level, which ranges from

fifty to sixty and has been set as the appropriate level for eleventh-grade students. Therefore, the readability

level of the English textbook published by Kemendikbud was low and did not match eleventh-grade students’

level.

1 INTRODUCTION

Reading is one of the skills that are acquired through

teaching and learning English in school. Reading

allows students to study writing and compose

sentences correctly. It is an easily identified language

skill since every activity in the classroom is tied to

reading. Students read not only to gather information

but also to improve their linguistic competence

(Harmer, 2011). In accord with Renandya and

Widodo (A Renandya, 2016), reading is the process

of collecting meaning from printed words, phrases,

paragraphs, or an entire written statement. It is the

process of looking at written words using our eyes

and compiling them to make perfect sense. If students

are given a reading text below their level, they will

not develop their reading ability. The lack of students’

reading ability can cause them to become poor

readers in the future. Therefore, Suitable reading

materials would enhance the students’ language

skills, whereas too simple text would tire the students,

and challenging material would frustrate them.

Furthermore, according to Nuttal in Berardo,

there are three things to consider in selecting texts:

suitability, exploitability, and one of them is

readability. Readability is a term that involves how

easy or difficult it is for students to comprehend and

understand a text. Some studies use the readability

formula to test the quality of textbook reading texts

because it is a very efficient and reliable way of

measuring the quality of textbooks (Yunita et al.,

2017).

Moreover, Martinc et al (Martinc et al., 2021)

imply that readability is the relationship of a text and

the working memory of a reader to grasp the meaning.

Sangia (Sangia, 2017) also states that readability is

frequently related to whether or not students read the

material. Readability is the study of written text

components that help or hinder the contexts of ideas

and information (Bailin and Grafstein, 2016). It can

be said that readability focuses on how difficult the

readers perceive the texts. It is a matter of choosing

suitable reading materials and how readers can

understand texts. Students reading abilities are also

affected by the readability level of a textbook. If

reading texts is not readable, students would move on

without getting information from the reading text.

Then they would not use their intelligence; instead,

they would lower their supposed competence.

Therefore, the readability level plays an essential part

Permatasari, M., Lathipatud Durriyah, T., Eviyuliwati, I., Deﬁanty, M., Sunengsih, N., Mahmud, N. and Wahyunengsih, .

The Readability Level of English Textbook Published by Kemendikbud for Grade Xi.

DOI: 10.5220/0011240200003376

In Proceedings of the 2nd International Conference on Recent Innovations (ICRI 2021), pages 31-36

ISBN: 978-989-758-602-6

in terms of providing a reading material that matches

students’ level.

The length of the sentences or the length of the

words determines readability. However, it is also

influenced by how informative and compelling the

text is written. Some experts or researchers interpret

the word readability in various ways. Readability is

how easily written materials may be read and

understood (Richards and Schmidt, 2010; Zulkifli,

2019). It is an essential part of readable material

which showcases the difficulty that lies in the text.

How the text goes beyond or within the level of the

readers is affecting the readability level. Readability

assessment is a way of identifying the appropriate

reading level for text (Deutsch et al., 2020).

Some approaches can be used to measure

readability. According to Alderson, there are two

approaches: cloze test and readability formulae. The

measuring of readability is carefully considered,

especially about the complexity of words and phrases.

In this case, the researchers used readability formulas.

Using mathematical calculations, they mainly focus

on the number of sentences, phrases, and easy and

difficult words that appear in a reading material. The

researchers used Rudolf Flesch, who developed the

Flesch Reading Ease formula. The formula by Flesch

Reading Ease score as follows:

𝑆𝑐𝑜𝑟𝑒 = 206.835 −

(

1.015 ×

𝐴

𝑆𝐿

)

− (84.6 ×

𝐴

𝑆𝑊)

(1)

Where:

Score = position on a scale of 0 (difficult) to 100

(easy)

ASL = average sentence length (the number of

words divided by the number of

sentences)

ASW = average syllable count per word (the

number of syllables divided by words)

It calculates reading ease on a scale from 1 to 100,

with 0 –30 being “very difficult,” 30 –50 being

“difficult,” 50 – 60 being “fairly difficult,” 60 –70

being “standard,” 70 –80 being “fairly easy,” 80 –90

being “easy” and 90 –100 being “very easy.”

To apply the formula, this study used the latest

version of the English textbook, published in 2017 by

the Ministry of Education and Culture for the

eleventh grade of Senior High School as the research

object. The government prepared the textbook in the

context of the implementation of the 2013

curriculum. Following the demands of the National

Curriculum, students are needed to explore and find

valuable learning sources without losing the cultural

context in their own country. The textbook provides

students with more communication activities and

involves the four language skills: listening, reading,

speaking, and writing. This also means that the

textbook is expected to help students improve their

language skills and expand their knowledge.

Following the background of the study above, the

researchers try to examine and describe the

readability level of the reading texts in the Ministry

of Education and Culture’s English textbook for the

eleventh grade of Senior High School. Using the

formula, the researchers can examine reading texts to

determine whether they are appropriate for students

or not. Thus, the study is titled The Readability of

English Textbook, published by Kemendikbud for

Grade XI.

2 METHOD

The study used the descriptive analysis method to

describe, elaborate, and analyse the readability level

of the English textbook 2017 Revised Edition for

eleventh-grade students of Senior High School

printed by the Ministry of Education and Culture.

This research described a phenomenon and its

characteristics, concerned with what happened rather

than how it happened (?).

The study used eleven reading texts which were

three transactional interaction texts, one analytical

exposition text, one explanation text, three narrative

texts, one hortatory exposition text, one speech text,

and one recount text (biography) in the English

textbook published by Kemendikbud for Grade XI.

The instrument of this study was the data analysis

procedures. Using the Flesch Reading Ease formula

as the readability measurement tool, the data were

collected by choosing reading texts from the English

textbook for Senior High School eleventh-grade

students 2017 updated edition, published by the

Ministry of Education and Culture. The researchers

then analyzed each reading text. After that, the

researchers then determined the readability level of

the selected reading texts using the Flesch Reading

Ease formula.

In order to obtain the result of the study. The data

analysis procedures applied by identifying the

reading texts in the textbook and counts the number

of syllables, words, and sentences in each text using

www.countwordsworth.com. The researchers

referred to this as a screening to identify reading texts

before applying the Flesch Reading Ease formula.

When using www.countwordsworth.com, the

procedures are carried out by copy and pasting the

texts.

ICRI 2021 - International Conference on Recent Innovations

In obtaining the study’s the calculation data, the

number of syllables, words, and sentences in each text

then applies the Flesch Reading Ease Formula. In

terms of data analysis, the study results were

described with the readability level of each reading

texts used in the English textbook published by

Kemendikbud for Grade XI. The findings will be used

to determine the difficulty level and reading grade

using the Reading Ease Scale of the Flesch Reading

Ease For mula, as stated in Dubay (2004) and shown

in Table 1.

Table 1: Reading Ease Scale by Flesch Reading Ease

Formula.

Reading Ease

Score

Difficulty Level

Estimated

Reading Grade

0 to 30 Very Difficult College Graduate

30 to 50 Difficult 13

to 16

grade

50 to 60 Fairly Difficult 10

to 12

grade

60 to 70 Standard 8

and 9

grade

70 to 80 Fairly Easy 7

grade

80 to 90 Easy 6

grade

90 to 100 Very Easy 5

grade

3 FINDINGS AND DISCUSSION

The research question focuses on the readability

level of the English textbook published by

Kemendikbud for Grade 11. Based on the results of

using the Flesch Reading Ease formula, the

researchers present the process of calculating and

presenting data in the research finding. In addition,

the researchers described the findings in the

discussion section. As the researchers analyzed the

reading texts, the results total 11 reading texts used

in this research. Firstly, the researchers listed the

reading texts; the following result was obtained as

the explanation below in Table 2:

Table 2: Reading Ease Scale by Flesch Reading Ease

Formula.

Title

Number

of Words

Number

Sentences

Number

Syllables

1 106 16 145

2 152 19 220

3 193 12 295

4 202 21 287

5 261 18 435

6 488 34 737

7 660 37 1146

8 750 43 1150

9 827 98 1201

10 1021 83 1318

11 1063 82 1276

Note: 1. Interpersonal transaction, 2.

Transactional Interaction, 3. Earthquake, 4.

Transactional Interaction Text (Personal Letter), 5.

Global Warming: Is it an end to our world?, 6.

President Sukarno of Indonesia: Speech at the

Opening of the Bandung Conference, April 18, 1955,

7. Life and times of Ki Hajar Dewantara, 8. Bullying:

A cancer that must be eradicated, 9. Vanity and Pride,

10. The Enchanted Fish, 11. The Last Leaf

Table 2 showed the number of sentences, words,

and syllables in each reading text. Then the data

above were calculated using the Flesch Reading Ease

formula. ASL (average sentence length) was counted

by dividing the number of words by the number of

sentences, while ASW (average number of syllables

per word) was counted by dividing the number of

syllables by the number of words. Then, the result of

the readability score of reading texts provided in the

English textbook for the eleventh-grade student of

Senior High School was presented in the following

Table 3:

Table 3: Readability Score of the Reading Texts Based on

Flesch’s Reading Ease Formula.

Readability

Score

Difficulty Level Reading Grade

84.2 Easy 6

grade

76.8 Fairly Easy 7

grade

58.2 Fairly Difficult 10

− 12

grade

76.9 Fairly Easy 7

grade

70.1 Fairly Easy 7

grade

64.6 Standard 8

− 9

grade

36.4 Difficult 13

− 16

grade

59.7 Fairly Difficult 10

− 12

grade

75.5 Fairly Easy 7

grade

85.2 Easy 6

grade

92.2 Very Easy 5

grade

Based on the readability score in the data analysis

above, two reading texts were classified into Fairly

Difficult level. The appropriate reading texts for the

eleventh grade of Senior High School are Text 3

(Earthquake), with a readability score of 58.2, and

Text 8 (Bullying: Cancer that must be eradicated)

with a readability score of 59.7. These two reading

texts were considered readable text for students who

are in grade 11.

The Readability Level of English Textbook Published by Kemendikbud for Grade Xi

Table 4: Readability Score of the Reading Texts Based on

Flesch’s Reading Ease Formula.

No Level

The Number

of Texts

Percentage

1 Very Easy 1 9.09%

2 Easy 2 18.18%

3 Fairly Easy 4 36.36%

4 Standard 1 9.09%

5 Fairly Difficult 2 18.18%

6 Difficult 1 9.09%

Table 4 showed the percentage of the readability

level from each reading text. Overall, the findings of

reading texts from reading one up to reading text 11

only two reading texts, which is 18.18%, are suitable

for the eleventh-grade students. It means that nine

texts are irrelevant to the students of Junior High

School of eleventh grade.

Based on the result of the study, there were two of

the reading text examined as the appropriate

readability level for eleventh grade students. The

result is similar to the findings of Miftahurrahmi and

Nababan (Miftahurrahmi et al., 2017; Nababan et al.,

2019). There were a few reading texts from the

textbooks found to be not suitable for students.

Contrary, the study conducted by Tabatabaei Bagheri

(Tabatabaei and Bagheri, 2013) discovered that the

reading texts that the researchers used had the most of

the reading texts in the appropriate readability level

for Iranian Senior High School Students. Tabatabaei

Bagheri claimed that the student’s background

knowledge and interest were involved because there

were students as the participant. The study conducted

by Uri Aziz (2018) also showed that all the reading

texts used from the English paper were appropriate

for Senior High School students. Students with low or

intermediate levels of reading skills were still able to

do the test, and at the same time, the texts were not

too easy but do not bore the students either.

Furthermore, the previous studies was also found

using another readability formula, it was Gunning

Fox. Also, the other difference was the object of the

textbook. The previous studies used textbooks from

private companies published textbooks, while in this

current study, the researchers used the Ministry of

Education and Culture published textbook.

Nevertheless, the readability of reading texts in the

previous studies was all conducted using the Flesch

Reading Ease formula. Therefore, this current study

uses the Flesch Reading Ease formula to examine the

readability level of reading texts in the English

textbook for senior high school eleventh grade. This

present study also aims to examine and describe the

readability level of reading texts in English textbook

published by the Ministry of Education and Culture

for the eleventh grade of Senior High School with a

descriptive analysis method. However, no studies

using the Ministry of Education and Culture

published textbook as the research object in the

previous studies. Therefore, this current study uses

the Ministry of Education and Culture published

textbook to describe the readability level of the

reading texts.

According to the result of this study using the

Flesch Reading Ease formula, the reading texts,

Earthquake and Bullying: Cancer that must be

eradicated, had the readability levels for eleventh

grade students. Those two reading texts indicated that

the words, sentences, and syllables were suitable for

students’ ability level to understand. Suitable reading

materials would enhance the students’ language

skills, whereas too simple text would tire the students,

and challenging material would frustrate them

(Zulkifli, 2019). Klare as cited in Dubai stated that the

factors influencing readability are word

characteristics such as the number of words,

concreteness, and nominal verb. And sentence

characteristics such as length of sentence, active or

passive sentence, and so on. The average sentence

length and syllables per word are the essential factors

that influence the readability level. Therefore, the

factors need to be understood by teachers as they

affected students’ reading ability. Above all, the

readability formula helps predict the readability of

texts in a short overview from semantic (vocabulary,

multiple syllables, and word length) and syntactic

(sentence length). Texts with slightly longer words

and longer phrases take longer to absorb, making

them more challenging to understand. These factors

could influence readability because it involves the

reading materials that are used by teachers, which in

this case, is the English textbook (Handayani et al.,

2021; Peng, 2015; Srisunakrua and Chumworatayee,

2019).

Although it is merely a rough measurement of

readability, the results are more objective since they

are established based on research and mathematical

calculations. The readability formulas’ scores lack a

conversion score that describes the academic

performance of EFL students in Indonesia, and they

are not always 100 percent accurate since they cannot

effectively measure students’ competency. It did,

however, become a disadvantage of the readability

formula. However, it is helpful for English teachers

who want to determine whether their reading text is

derived from actual material or a textbook acceptable

for their student’s level of competency. As a result,

one thing to consider is that the application of the

ICRI 2021 - International Conference on Recent Innovations

readability formula must be followed by their

subjective evaluation, alignment to the learning

objectives, and the current Senior High School

curriculum.

However, the rest of the reading texts in the

English textbook were not suitable for students

because they had more challenging words, sentences,

and syllables. Because of that reason, textbooks are

seen as a less fascinating learning resource than

others, such as PowerPoint, Videos, or Audio. There

are various drawbacks to reading textbooks. For

example, poorly chosen material can be tremendously

frustrating for learners since they will make it

challenging; determining the validity of resources is

difficult; texts are frequently too high. Moreover,

concerning the suitable and engaging material, it

should be considered by teachers because of the

importance of providing reading texts with the

readability level as students’ level, in line with

Petunjuk Teknis Pedoman Buku yang digunakan oleh

Satuan Pendidikan dalam Peraturan Menteri

Pendidikan dan Kebudayaan Nomor8 Tahun 2016

tentang Buku yang Digunakan oleh Satuan

Pendidikan (Knight, 2015), “Materi pengetahuan

yang diinformasikan melalui Buku Teks Pelajaran

dan Buku Non Teks Pelajaran sangat penting. Oleh

karena itu penyajian materi harus ditata dengan

menarik, mudah dipahami, memiliki tingkat

keterbacaan yang tinggi” As textbooks are always an

important learning source, it is also defined as

technologically organized, structured textual and

visual information.

4 CONCLUSIONS

In conclusion, this research has proved that the

readability level of the textbooks that the school or

government prescribes for teachers and students to

use needs to be considered. As a result, the readability

effectiveness must be checked by teachers. The

results obtained from the Flesch’s Reading Ease

(FRE) formula utilized in this research showed that

only two reading texts or 18.18% of the reading texts

that exist in the textbook were appropriate for Senior

High School eleventh-grade students.

Following the conclusion earlier, the researchers

would like to deliver some suggestions to teachers,

the researchers would like to give some suggestions

to teachers, which the researchers hope that teachers,

particularly English teachers, analyze the readability

level of reading texts in textbooks before utilizing

them in the classroom. Teachers cannot assume that

all reading texts in the textbook they are using are at

the appropriate readability level for the student’s

level. Good textbooks are excellent teaching

materials. They are helpful to both teachers and

students. Furthermore, the researchers recommend

that English teachers who utilize this English

textbook, 2017 revised edition, published by the

Ministry of Education and Culture for the eleventh

grade of Senior High School as a learning source or

media in the classroom use just the two specific

reading texts.

Furthermore, for other researchers, the

researchers hope that this research will provide a

guideline for future researchers who want to do

similar studies on the readability of reading texts,

particularly in English textbooks. Finding out how

reading texts in school textbooks is connected to how

students would understand the reading texts they read

is essential. The researchers also encourage future

researchers to study the readability level of reading

texts in Ministry of Education and Culture textbooks

or textbooks from several other publishers for the

same or other grades or subjects.

REFERENCES

W. H. Dubay, The Principles of Readability. Impact

Information, 2004.

J. Harmer, How to Teach Writing. Longman, 2007.

W. A. Renandya and H. P. Widodo, English Language

Teaching Today: Linking Theory and Practice.

Switzerland: Springer Nature, 2016.

A. Ismail, N. Yusof, and K. Yunus, “The readability of

Malaysian English children books: A multilevel

analysis,” Int. J. Appl. Linguist. English Lit., vol. 5, no.

6, pp. 214–220, 2016.

S. A. Berardo, “The Use of Authentic Materials in the

Teaching of Reading,” Read. Matrix An Int. Online J.,

vol. 6, no. 2, pp. 60–69, 2006.

L. R. Yunita, S. Suharsono, and A. Munir, “Readability in

Classroom Practice of Report Text as Reading Material

of Eleventh Graders,” J. English Lang. Lit., vol. 8, no.

3, p. 673, 2017.

M. Martinc, S. Pollak, and M. Robnik-Šikonja, “Supervised

and Unsupervised Neural Approaches to Text

Readability,” Comput. Linguist., vol. 47, no. 1, pp.

141–179, 2021.

R. A. Sangia, “Assessing Reading Text in English

Textbook for Eleventh Grader Published By Ministry

of Education and Culture,” Paramasastra, vol. 3, no. 1,

2017.

A. Bailin and A. Grafstein, Readability: Text and Context,

1st editio. United Kingdom: Palgrave Macmillan, 2016.

J. C. Richards and R. Schmidt, Longman Dictionary of

Language Teaching and Applied Linguistics, Third.

Pearson Education Limited, 2002.

Zulkifli, “Disparity among Indonesian Sociology of

The Readability Level of English Textbook Published by Kemendikbud for Grade Xi

Education textbooks,” J. Educ. Media, Mem. Soc., vol.

11, no. 2, pp. 86–104, 2019.

T. Deutsch, M. Jasbi, and S. Shieber, “Linguistic Features

for Readability Assessment,” pp. 1–17, 2020.

J. C. Alderson, “Assessing Reading.” Cambridge

University Press, 2000.

S. B. Mirshojaee and R. Sahragard, “Reading

Comprehension Passages of Iranian General English

Books and MA,” Shiraz University of Iran, 2015.

N. F. M. Uri and M. S. A. Aziz, “Assessing Readability of

a National Exam Reading Texts in Malaysia,” Eur. J.

English Lang. Teach., vol. 4, no. 1, pp. 149–164, 2018.

H. Nassaji, “Qualitative and descriptive research: Data type

versus data analysis,” Lang. Teach. Res., vol. 19, no. 2,

pp. 129–132, 2015.

Miftahurrahmi, H. Syarif, and Fitrawati, “The Readability

of Reading Texts in "Look Ahead: An English Course”

Textbook for Senior High School,” English Language

Teaching Study Program of FBS Universitas Negeri

Padang, 2017.

L. U. A. Nababan, S. D. C. Marpaung, and F. S.

Simangunsong, “The Readability in English Zone

Textbook for the Tenth Grade Students of Sma N 2

Bandar,” Aicll Annu. Int. Conf. Lang. Lit., vol. 2, no.

1, pp. 10–19, 2019.

E. Tabatabaei and M. S. Bagheri, “Readability of Reading

Comprehension Texts in Iranian Senior High schools

Regarding Students’ Background Knowledge and

Interest,” J. Lang. Teach. Res., vol. 4, no. 5, pp. 1028–

1035, 2013.

R. Handayani and F. M. Ivone, “The Readability Level of

Reading Texts in Erlangga Straight Point Series :

English for Eleventh Grade Students,” J. Pendidik., vol.

5, no. 11, pp. 1579–1587, 2020.

C.-C. Peng, “Textbook Readability and Student

Performance in Online Introductory Corporate Finance

Classes,” J. Educ. Online, vol. 13, no. 2, pp. 35–49,

2015.

T. Srisunakrua and T. Chumworatayee, “Readability of

Reading Passages in English Textbooks and the Thai

National Education English Test: A Comparative

Study,” Arab World English J., vol. 10, no. 2, pp. 257–

269, 2019.

D. A. Gunantar, “Textbooks Analysis: Analyzing English

As a Foreign Language (EFL) Textbooks From the

Perspective of Indonesian Culture,” Lang. Circ. J.

Lang. Lit., vol. 11, no. 2, pp. 173–182, 2017.

S. Ahmed, “Authentic ELT Materials in the Language

Classroom : An Overview,” J. Appl. Linguist. Lang.

Res., vol. 4, no. 2, pp. 181–202, 2017.

“Peraturan Menteri Pendidikan Dan Kebudayaan Nomor 8

Tahun 2016 Tentang Buku Yang Digunakan Oleh

Satuan Pendidikan.” pp. 1–22, 2016.

B. A. Knight, “Teachers’ use of textbooks in the digital

age,” Cogent Educ., vol. 2, no. 1, 2015.

ICRI 2021 - International Conference on Recent Innovations