Development and Validation of Unidimensional Psychological Scales:

A Scalogram Analysis Approach

Qudsiya Jamal Khader Basha

1,*

, Murugan P. P

and Mahandrakumar K

Freelance Researcher & Educator, Extension Education, Texas, U.S.A.

Extension Education, TNAU, Coimbatore, India

Department of Agrl. Extension and Rural Sociology, AC & RI, Kudimiyanmalai, India

Keywords: Guttmann Scale, Scalogram Technique, Psychological Scale, Coefficient of Scalability, Coefficient of

Reproducibility, Test-Retest Reliability, Concurrent Validity, Extension Research.

Abstract: This research article presents a comprehensive methodology for developing and validating unidimensional

psychological scales using the Scalogram technique, with a particular focus on the Guttmann Scale. The study

operationalizes key constructs such as Economic Motivation, Scientific Orientation, and Risk Orientation,

offering a nuanced understanding of farmers' attitudes toward adopting new ideas and technologies in

agriculture. The procedure involves meticulously identifying and scrutinizing items through literature analysis

and expert consultation. The article details the step-by-step process of Scalogram analysis, encompassing the

calculation of the Coefficient of Reproducibility (CR) to assess unidimensionality and the Coefficient of

Scalability (CS) to evaluate scalability. The study emphasizes the importance of achieving a CR of 0.90 or

higher and a CS of 0.60 or higher for a scale to be considered acceptable. The analysis results, including error

minimization and deviation from perfect reproducibility, contribute to the refinement of the scales. Reliability

and validity of the developed scales are established through the Test-Retest method for reliability and

concurrent validity through correlation with existing psychological variables. The final scales are standardized

using established scoring procedures. The article concludes with insights into the administration and scoring

of the finalized scales, providing a comprehensive guide for researchers and practitioners interested in robust

psychological scale development. This research contributes to the Extension research field by offering a

systematic and statistically sound approach to understanding and measuring complex psychological constructs

within the context of agricultural decision-making.

1 INTRODUCTION

The development of psychological scales is a crucial

endeavor in understanding and quantifying complex

human attitudes and behaviors. One method that has

gained prominence in this process is the Guttman

Scale, pioneered by Louis Guttman.

It is a cumulative unidimensional scale, “A

unidimensional scale is characterized by a pattern

where endorsing the item representing the extreme

position leads to endorsing all items that are less

extreme as well.”

This technique developed by Louis Guttman

commonly known as Scalogram analysis involves

presenting a series of statements to which a

respondent indicates their level of agreement or

Corresponding author

disagreement. (Ray and Mondal, 2011), allowing for

the orderly arrangement of items along a continuum.

This approach offers a systematic and rigorous

methodology for creating unidimensional

psychological scales, ensuring that responses align

with a distinct pattern.

2 METHODOLOGY

The brief procedure for development of the scales

using Guttmann’s scalogram analysis is detailed as

follows.

This study operationalizes and measures key

psychological constructs. “A scale serves as a tool for

measuring a particular attribute or dimension. Scaling

Basha, Q. J. K., P. P, M. and K, M.

Development and Validation of Unidimensional Psychological Scales: A Scalogram Analysis Approach.

DOI: 10.5220/0012882600004519

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st International Conference on Emerging Innovations for Sustainable Agriculture (ICEISA 2024), pages 103-109

ISBN: 978-989-758-714-6

103

techniques are employed to arrange a series of items

in a sequential order along an orderly defined

continuum.” (Ray and Mondal, 2011).

Table 1: Initial scales developed for testing and validation.

Sl. No Statements Yes No

I Economic Motivation

You do agriculture

1. To become rich and have a luxurious lifestyle

2. To have a decent living

3. To sustain my livelihood

4. Since I have nothing to do other than agriculture

5. Since I have known nothing other than agriculture

6. Since others are doing

II Scientific Orientation

What farming method you would prefer/wish to do

1. Remote/ mobile based farming method

2. Automated farming method

3. Motorized farming method (tractors, fuel/ power operated

machineries)

4. Manual faming (tools and implements)

III Risk orientation

If you were provided with credit assistance from bank which farming, you would prefer

1. Export business-oriented farming

2. Value addition and food processing

3. Organic farming and marketing of organic certified

products

4. Seed production-oriented farmin

5. Conventional crop production

ICEISA 2024 - International Conference on ‘Emerging Innovations for Sustainable Agriculture: Leveraging the potential of Digital

Innovations by the Farmers, Agri-tech Startups and Agribusiness Enterprises in Agricu

104

Items to measure the economic motivation,

scientific orientation, and risk orientation of farmers

were identified through thorough literature analysis.

Furthermore, expert discussions with extension

specialists from the Department of Agricultural

Extension and Rural Sociology of TNAU and

Madurai were conducted for scrutiny. Thus, through

literature analysis and expert consultation, items are

carefully identified and scrutinized to capture the

nuances of the targeted constructs.

2.1 Economic Motivation

It is operationalized in terms of an individual's

prioritization of economic goals and the willingness

for profit maximization.

2.2 Scientific Orientation

It is operationalized as the extent to which a farmer is

inclined toward utilizing scientific methods in

agricultural and allied practices.

2.3 Risk Orientation

Risk orientation pertains to the extent to which

individuals are inclined towards taking up risk and

uncertainty with the courage to handle existential

challenges. The goal was to establish an ordinal value

for each scale using the selected items and to use the

value in a variety of statistical analyses.

Based on the preliminary discussion items were

selected and ordered for each scale to be developed.

The developed scales are to be calculated for its

coefficient of reproducibility and coefficient of

scalability for standardization.

2.4 Calculation of Co-Efficient of

Reproducibility

The complete list of items, arranged in a simple

yes/no format, was presented to 30 farmers in a non-

sample area via a survey. Each respondent indicated

their agreement or disagreement with each item. The

data were organized into a matrix where rows

represented respondents and columns represented

items, with entries of ones and zeros denoting

agreement or disagreement with each item,

respectively.

In assessing errors of inclusion and omission

within a Guttman Scale, two methods were typically

employed. The first, proposed by Guttman (1950), is

known as the minimization of error approach. It

involves counting the minimum number of responses

that need to be altered to transform a respondent's

response pattern into an ideal scale. Here, the ideal

scale reflects the order of items and doesn't consider

the total number of items a respondent may have.

The second method, deviation from perfect

reproducibility, is more conservative. It determines

errors based on an ideal response pattern considering

both the order of responses and the total number of

items a respondent possesses, as described by

Goodenough (1944) and Edwards (1983). ‘

The coefficient of reproducibility (CR) for each

scale is derived from this method, serving as a

measure of the unidimensionality of the items within

the scale.

𝐶𝑅 = 1 −

𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑟𝑟𝑜𝑟𝑠

𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑒𝑟𝑟𝑜𝑟𝑠

(1)

The CR is calculated using a specific formula. The

Coefficient of Reproducibility (CR) assesses the

degree of unidimensionality exhibited by the items

within the scale and it is calculated by using the given

formula.

As per Guttman's measure, a scale is deemed

acceptable if it possesses fewer than 10 percent

erroneous entries. Therefore, a coefficient of

reproducibility (CR) equal to or exceeding 0.90 is

considered evidence that a set of items is

unidimensional in its scaling.

2.5 Calculation of Coefficient of

Scalability

The Coefficient of Reproducibility (CR) has a

limitation in that it is influenced by extreme marginal

distributions both in terms of items and individuals,

which means that a high CR can be achieved even

with random responses of the sample respondents

(Menzel, 1953; McIver and Carmines, 1981).

For instance, if an individual randomly responds

"yes" to 90 percent of the items on a list, it becomes

relatively easy to predict whether this individual has

a "yes" for any given item based solely on this fact.

This phenomenon is referred to as the

extremeness of individuals. Similarly, if 90 percent of

farmers respond "yes" to a particular item, predicting

whether any given individual has this item becomes

rather straightforward, within a 10 percent margin of

error, without any additional information.

This scenario is known as the extremeness of

items. In either case, accurate predictions of data

arrangement can be made simply by using the

category with the highest frequency (i.e., the modal

category).

Development and Validation of Unidimensional Psychological Scales: A Scalogram Analysis Approach

105

Therefore, although the data may exhibit

relatively few scale errors, resulting in a high CR,

they may not necessarily reflect scalability or

departure from randomness. Scalability implies that

categories and individuals can be meaningfully

arranged from highest to lowest, and the ability to

predict order solely based on marginals undermines

such meaningfulness.

Menzel (1953) suggested that the degree of

success in reproductions is influenced by three

factors: (1) the extremeness of items, (2) the

extremeness of individuals, and (3) the scalability of

the items for the given individuals. Therefore, to

determine if the data truly exhibit scalability, it is

necessary to control for extreme responses.

To address this issue, Menzel (1953) used the

Coefficient of Scalability (CS), which measures the

predictability of the scale relative to the level of

prediction achieved solely by considering the row and

column marginals. The formula is given as follows.

𝐶𝑆 = 1 −

𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑐𝑎𝑙𝑒 𝑒𝑟𝑟𝑜𝑟𝑠

𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑚𝑎𝑟𝑔𝑖𝑛𝑎𝑙 𝑒𝑟𝑟𝑜𝑟𝑠

(2)

Marginal error refers to the count of non-modal

frequencies within the obtained dataset. When the

proportion of marginal errors compared to total errors

is higher, it increases the CS. As the scale exhibits

fewer errors than anticipated by chance, the CS

approaches 1.0. Menzel recommends a CS of 0.60 or

above as acceptable.

The initial analysis of the selected items for risk

orientation and economic motivation produced a

respectable CR of 0.83, CS of 0.23 and CR of 0.87,

CS of 0.58 respectively but, hoping to achieve a CR

of 0.90 and CS of 0.60, various deletions was tried.

There are several ways to do these deletions, but the

easiest is to look for the item with the most errors.

By removing certain items, a higher CR and CS

can be achieved. In the present study, a CR of 0.900,

a CS of 0.643, with three items for economic

motivation, a CR of 0.907, a CS of 0.655, with three

items for scientific orientation, a CR of 0.947, a CS

of 0.821 using three items for risk orientation were

achieved for the final scale.

Table 2: Coefficients of Scalability and Reproducibility of

the developed scales.

Results

Coefficient of

Scalabilit

Coefficient of

roducibilit

Scales CS CR

Economic

motivation

0.643 0.900

Scientific

orientation

0.655 0.907

Risk

orientation

0.821 0.947

2.6 Reliability of the Scales Developed

The developed scales were further standardization

through the establishment of their reliability.

According to Kerlinger and Lee (2000), reliability

pertains to the accuracy or precision of a measuring

instrument. To assess the reliability of the attitude

scale, the Test-Retest method was employed. Validity,

which essentially denotes truthfulness, refers to "the

degree to which a test measures what it claims to

measure" (Ray and Mondal, 2011).

2.6.1 Test-Retest Reliability

The test-retest method involves administering the

developed scale twice and then computing the

reliability coefficient between the two sets of test

scores. Therefore, the developed scales were

administered to the farmers with a fortnight interval,

the significance of the correlation was achieved

known as the reliability index.

2.7 Concurrent Validity

Concurrent validity was utilized to gauge the validity

of the scale. Concurrent validity was established by

examining its correlation with a criterion that is

currently available.

In this study, scores on the newly constructed

scales measuring economic motivation, scientific

orientation, and risk orientation were correlated with

scores obtained from existing scales of psychological

variables developed by Supe (1969).

2.7.1 Standardized Scale for Validity

Each scale consisted of six statements, incorporating

a mix of positive and negative items to capture

nuanced responses from participants.

ICEISA 2024 - International Conference on ‘Emerging Innovations for Sustainable Agriculture: Leveraging the potential of Digital

Innovations by the Farmers, Agri-tech Startups and Agribusiness Enterprises in Agricu

106

For detailed descriptions of these scales and their

individual items, please refer to Appendix 1 of this

study, where each scale is provided along with

instructions for administration and scoring. The

operationalization of the scales are as follows. The

economic motivation was assessed using a scale

developed by Supe (1969). Each statement was rated

on a five-point continuum, ranging from strongly

agree to strongly disagree. The scale comprised six

statements, with one being negative and the

remaining being positive statements. The scores for

each item were summed to determine the economic

motivation score for each respondent, which ranged

from 6 to 42.

Supe’s (1969) Scientific Orientation Scale was

employed in the study. This scale comprised six

statements, with one statement being negative and

the others positive. The scores from each item were

summed to determine the scientific orientation score

of each respondent, which fell within a range of 6 to

42.

The measurement of risk preference utilized

Supe's (1969) Risk orientation Scale. This scale

comprised six statements, with two being negative

and the remainder positive. The scores for each item

were summed to calculate the risk orientation score

for each respondent, ranging from 6 to 42. According

to Singh (1977), the resulting correlation coefficient

serves as an indicator of concurrent validity.

Therefore, both the newly constructed scales and the

standard scales were administered to 30 farmers from

a non-sample area. Using SPSS, the Pearson Product

Moment correlation coefficient (r) was calculated for

each scale. The significant correlation observed

serves as a measure of the concurrent validity of the

developed scales which is presented in table 3.

Table 3: Validity and reliability of the developed scales.

Results

Validity

(Concurrent

validity)

Reliability

(Test-retest

reliability)

Scales

Pearson Product

Moment Correlation (r)

Reliability Index

Economic

motivation

0.533** 0.96 **

Scientific

orientation

0.633** 0.76 **

Risk

orientation

0.457* 0.79 **

*Significance at 5% level

** Significance at 1% level

2.8 Scoring of Final Scales

Based on the coefficient of reproducibility and

scalability the items were ranked for each scale in the

descending order. The final scale adopted is presented

in Table 4 along with the scoring of the individual

scales developed.

Table 4: Final Scales developed using Scalogram approach.

Economic motivation

You do agriculture

Sl.

Statements Score

1. To become rich and have a luxurious

lifestyle

2. To have a decent living 2

3. To sustain my livelihood 1

Scientific Orientation

What farming method you would prefer/wish to do

1. Remote/ mobile based farming method 3

2. Motorized farming method (tractors,

fuel/ power operated machineries)

3. Manual faming (tools and implements) 1

III

Risk orientation

If you were provided with credit assistance from bank

which farming, you would prefer

1. Export business-oriented farming 3

2. Value addition and food processing 2

3. Conventional crop production 1

3 CONCLUSIONS

The Scalogram analysis approach is particularly

valuable in capturing the intricacies of human

attitudes, as the Guttman Scale Scalogram method

emphasizes the need for responses to follow a clear

pattern, endorsing fewer extreme items if the most

extreme item is endorsed.

The research process involves refining the scales

based on analysis results, addressing errors, and

aiming for the desirable CR and CS thresholds for

Development and Validation of Unidimensional Psychological Scales: A Scalogram Analysis Approach

107

scale acceptability. This approach, detailed in the

article, ensures the unidimensionality and scalability

of the developed scales.

The calculation of the Coefficient of

Reproducibility and Coefficient of Scalability helps

in assessing the reliability and validity of the scales,

with thresholds of 0.90 and 0.60 respectively

indicating acceptability.

Furthermore, the study establishes reliability

through the Test-Retest method and concurrent

validity through correlation with existing

psychological variables. The standardized scoring

procedures facilitate ease of administration and

interpretation of the scales. Overall, this research

contributes significantly to the Extension research

field by offering a systematic and statistically sound

approach to measuring complex psychological

constructs in the context of agricultural decision-

making.

The Guttman Scale Scalogram approach provides

researchers with a robust and statistically sound

framework to delve into the complexity of

psychological constructs, offering valuable insights

into human attitudes and behaviors. These reliable

and valid psychological scales that can be utilized for

various research and practical applications.

REFERENCES

Edwards, A. L. (1983). Techniques of attitude scale

construction: Ardent Media.

Goodenough, F. L. (1944). Expanding opportunities for

women psychologists in the post-war period of civil and

military reorganization. Psychological Bulletin, 41(10),

706.

Guttman, L. (1950). The basis for scalogram analysis.

Menzel, H. (1953). A new coefficient for scalogram

analysis. Public Opinion Quarterly, 17(2), 268-280.

McIver, J., and Carmines, E. G. (1981). Unidimensional

scaling (Vol. 24): Sage.

Kerlinger, F., and Lee, H. (2000). Foundations of

behavioral research, United States of America:

Wadsworth Thomson Learning. In: Inc.

Singh, K. K., Patrin. M. L. and H. C. Sharma. 2000.

Environmental protection and sustainable agriculture.

Intensive agriculture, 38 (1-2): 12-16.

Supe, S. V. (1969). Factors related to different degrees of

rationality in decision making among farmers. IARI,

Division Of Agricultural Extension; New Delhi.

Ray, G., and Mondal, S. (2011). Research methods in social

sciences and extension education: Kalyani Publishers.

APPENDIX

Table 4: Standardized scales developed by Supe (1969) used for reliability testing.

I Economic motivation

Sl.No. Statement SA A UD DA SDA

1 A farmer should work towards larger yields and

economic benefits

2 The most successful farmer makes most profit.

3 A farmers should try new farming methods

4 A farmer should grow HYVs to make good profit.

5 It is difficult for farmer’s children to make good start

unless he provides them with economic assistance.

6* A farmer should earn his living but the most

important thing in life can’t be defined in economic

terms.

II Scientific Orientation

1 New methods of farming give better results to a

farmer than old methods

ICEISA 2024 - International Conference on ‘Emerging Innovations for Sustainable Agriculture: Leveraging the potential of Digital

Innovations by the Farmers, Agri-tech Startups and Agribusiness Enterprises in Agricu

108

2* The way of farmers’ forefathers farmed is still the

best of way to farm today

3 Even a farmer with lots of experience should use new

methods of farming

4 A good farmer experiments with new ideas in

farming

5 Though it takes time for a farmer to learn new

methods in farming it is worth the efforts

6 Traditional methods of farming have to be changed in

order to raise the level of living of a farmer

III Risk orientation

1* A farmer should grow more number of crops to avoid

greater risks involved in growing one / two crops.

2 A farmer should take more of chance in making a big

profit to be constant with smaller but less risky profits.

3 A farmer who is willing to take greater risks than the

average farmer actually does better financially

4 It is good for a farmer not to take risk when he known his

chance of success is fairly high.

5* It is better for a farmer not to try new farming methods

unless mostly other farmers have used it with success

6 Trying an entirely and new method in farming by a

farmer involved risks but it is worth.

*Negative statements

SA – Strongly agree, A- Agree, UD- Undecided, DA- Disagree, SD – Strongly disagree

The scoring procedure followed for the above standard scales is as follows.

Response

Strongly

Agree

Agree Undecided Disagree

Strongly

Disagree

Positive statements 7 5 4 3 1

Negative statement 1 3 4 5 7

Development and Validation of Unidimensional Psychological Scales: A Scalogram Analysis Approach

109