THE SHORT-TERM BENEFITS OF EDUCATIONAL ROBOTICS

WHEN PAIRED WITH GEOSPATIAL TECHNOLOGIES IN

INFORMAL LEARNING ENVIRONMENTS

Bradley S. Barker, Gwen Nugent, Viacheslav I. Adamchuk

Institute of Agriculture and Natural Resources, University of Nebraska-Lincoln, 114 Ag Hall, Lincoln, NE, U.S.A.

Neal Grandgenett

Teacher Education Department, University of Nebraska-Lincoln, Omaha, NE, U.S.A.

Keywords: Informal Learning, STEM, Robotics, GPS, GIS.

Abstract: Educational robotics, when paired with geospatial technologies and taught in an informal educational

environment, can be an innovative strategy to teach youth about science, technology, engineering, and

mathematic (STEM) concepts. However, little is known about the true effects on conceptual knowledge and

associated attitudes. Therefore, this study was conducted to examine the short-term effects of a series of

five-day summer robotics/geospatial camps held in Nebraska. The study was conducted at six diverse

locations and consisted of a five-day 4-H camp experience. The study examined the experiences of 147

youth between the ages of 10 and 15. A pretest-posttest quasi-experimental design was used in the study.

Instrumentation consisted of a 37-question multiple-choice assessment targeting various STEM topics, and a

38-question attitude questionnaire assessing STEM interests and attitudes. Results suggest that the 4-H

robotics and geospatial summer camp program is a promising approach for supporting STEM-related

learning and enhancing attitudes towards STEM.

1 INTRODUCTION

The Nebraska 4-H, with grant funding from the

National Science Foundation, is developing a

program to increase science, technology,

engineering and mathematics (STEM) achievement

and interest using robotics and geospatial

technologies. The widespread availability of robotic

kits such as the LEGO NXT Mindstrom kit,

handheld Global Positioning System (GPS) devices,

and geographical information systems (GIS) like

GoogleEarth and ArcMap make it possible for youth

(ages 10 to 15) to explore the integration of these

technologies. The curriculum was developed by

Nebraska 4-H and faculty from University of

Nebraska’s Biological Systems Engineering

Department, in cooperation with Carnegie Mellon

University’s Robotics Academy, and involves 40

hours of instruction. The 40-hour summer camp

activities include the building and programming of

robots, working with handheld GPS receivers to

explore and collect information, and the

development and customization of GIS maps. The

camp activities were led by project staff and in some

cases faculty from the University of Nebraska. The

content and context for the activities were delivered

in a short introductory lecture format followed by

hands-on activities. Two formal STEM-related

assessments were also administered, one related to

conceptual learning and one related to attitudes. The

participating youth also shared their general

perceptions of the activities.

Research in the use of educational robotics in an

informal learning environment implies that robotics

can increase academic achievement in specific

STEM concept areas closely aligned with formal

education topics and coursework (Nourbakhsh et al.

2005; Barker & Ansorge, 2007; Barker, Nugent,

Grandgenett, & Hampton, 2008). Similarly, past

research has indicated that GIS can be used to teach

project-based science, environmental education and

geography concepts to middle school students

(McWilliams & Rooney, 1997). Research also sug-

393

S. Barker B., Nugent G., I. Adamchuk V. and Grandgenett N. (2010).

THE SHORT-TERM BENEFITS OF EDUCATIONAL ROBOTICS WHEN PAIRED WITH GEOSPATIAL TECHNOLOGIES IN INFORMAL LEARNING

ENVIRONMENTS.

In Proceedings of the 2nd International Conference on Computer Supported Education, pages 393-398

 SciTePress

Table 1: Camp Participant Demographics.

gests that the use of GIS helps in the development of

analytical skills and problem solving (Wanner &

Kerski, 1999). Moreover, there is growing interest in

examining students’ attitudes towards learning, with

recognition that affect surrounds cognition and can

moderate learners’ conceptual change (Alsop &

Watts, 2003; Koballa & Glynn, 2007). Measuring a

student's attitude is not a trivial matter; and much of

the robotics literature looking at attitudes relies

heavily on subjective secondary observation. For

example, Rogers and Portsmore (2004) reported that

using robotics as an outreach activity in elementary

schools increased confidence and interest in

mathematics and science. This conclusion was based

on teacher perceptions; the researchers did not

directly gather any data from individual youth

participants.

There are several instruments that have been

developed to assess youth attitudes within science-

related contexts. The most widely used is arguably

the Scientific Attitude Inventory (SAI) developed by

Moore and Sutman (1970). The SAI (I) consisted of

a 60 item, four-point Likert-type scale with a series

of 12 statements of attitude called “position

statements.” These twelve position statements assess

six scientific attitudes – three based on intellectual

attitudes and three based on emotional attitudes. The

six attitudes include: 1) laws of science, 2) scientific

explanation, 3) manner of scientific observation, 4)

value of scientific activities, 5) usefulness of science

to society and 6) student career aspirations. Each

scientific attitude has a positive and a negative scale

to create the 12 position statements used as

potentially measurable constructs. According to

Moore and Sutman (1970) the reliability of the SAI

was measured through the use of the Winer test-

retest method using the pre and posttest scores of the

control group resulting in a test-retest reliability

coefficient of .934. Based on the lack of significance

using the SAI and other instruments when piloting

the project, we elected to develop our own

instrument based on specific constructs that

originate from our 4-H robotics and GPS/GIS

program. Our instrument measures eight scales

including: task values in science, mathematics,

robotics, and GPS/GIS, problem solving/critical

thinking, cooperative learning/teamwork, self

efficacy in robotics, and self efficacy in GIS/GPS.

2 PURPOSE AND

METHODOLOGY

The purpose of this study was to investigate the

short-term impacts of informal summer programs

centered on robotics and geospatial technologies in

a) promoting STEM learning for youth ages 10-15

and b) positively impacting their attitudes towards

STEM. A total of 147 participants in six different 4-

H facilitated camps participated in the summer

program. Overall, 112 males and 35 females

attended the camps. In addition, 75% of participants

were identified as Caucasian, 12% were African

American, 12% were Hispanic and 1% Asian. The

overall mean age for the camps was 12.28 years with

a median age of 12.00 years. Demographics are

displayed by location in table one.

2.1 Instrumentation

The instrumentation used for the study consisted of

two parts. To measure STEM learning, the project

staff developed a 37-item, paper-and-pencil,

multiple-choice assessment, covering a variety of

topics including computer programming,

mathematics, geospatial concepts and engineering/

robotics. The assessment was based on a previous

24-item robotics assessment instrument that

demonstrated a Cronbach’s alpha reliability

Location

Demographics of Camp Participants

N total

Male

Female

Age (Mean)

% Minority

Overnight

1. Omaha North

11.39

100

2. Omaha South

11.12

3. Lincoln

12.52

Yes

4. Ord

12.40

5. Chadron

12.69

Yes

6. Grand Island

12.80

CSEDU 2010 - 2nd International Conference on Computer Supported Education

394

Table 2: Content questionnaire paired samples test.

Location

Paired Differences

Pre Mean

Post Mean

Mean

Std.

Deviation

Sig. (2-tailed)

Omaha North

10.80

10.53

.267

3.22

.32

.753

Omaha South

11.58

15.50

-3.92

3.55

-3.82

.003

Lincoln

16.87

20.87

-4.00

3.01

-10.90

.001

Ord

17.60

23.80

-6.20

4.21

-4.66

.001

Chadron

16.53

23.20

-6.67

2.90

-8.93

.001

Grand Island

15.89

23.78

-7.89

3.18

-10.53

.001

coefficient of 0.86 (Barker & Ansorge, 2007). Two

experts from Carnegie Mellon University’s Robotics

Academy and two engineers from the University of

Nebraska-Lincoln Department of Biological

Systems Engineering reviewed and validated the

assessment instrument’s content. The overall

Cronbach’s alpha reliability coefficient of 0.80 was

reported for this instrument.

The attitude instrument was also developed by

the project staff and was modeled after the

Motivated Strategies for Learning Questionnaire

(Pintrich, et al., 1991). The questionnaire focuses on

the following eight constructs: task value for

science, mathematics, robotics and GPS/GIS,

problem solving/critical thinking, cooperative

learning/teamwork, self-efficacy in robotics and

self-efficacy in GPS/GIS. All the statements on the

attitude instrument used positively worded items due

to the relatively young age of the participants. The

task value for science included questions like “It is

important to me to learn how to conduct a scientific

investigation.” The mathematics task value construct

included questions like “It is important for me to

learn how to make accurate measurements to help

solve mathematical problems.” The robotics

construct asked questions like “It is important for me

to learn about robotics.” The GPS/GIS construct

included questions like “It is important for me to

learn about GIS.” In addition, problem solving skills

(i.e.” I try new methods to solve a problem when

one does not work”) and teamwork constructs (i.e. “I

like being part of a team that is trying to solve a

problem”) were also explored. Finally the instrument

examined self-efficacy in robotics (i.e. “I am

confident that I can program a LEGO robot to follow

a black line using a light sensor”) and GPS/GIS

concepts (i.e. “I am certain that I can log locations of

a series of waypoints within a GPS unit”). The

overall Cronbach’s alpha reliability coefficient of

0.94 was reported for this administration of the post

attitudinal instrument.

2.2 Data Collection

The pretest was administered by the researchers on

the first day of the camp prior to the start of program

activities. The posttest was administered in the

morning of the last day of camp. Administration of

the pretest-posttest assessment was conducted in the

same manner for each camp.

2.3 Data Analysis Procedures

The study used a pretest-posttest quasi-experimental

design, with the same assessment acting as both a

pretest and posttest in each summer campsite. The

learning assessment used a total score for the

number of items correct. The primary analysis was a

repeated measures t-test for the combined groups by

location. The attitudinal instrument used a five-

point Likert-type scale with five equaling “strongly

agree” and one equaling “strongly disagree”. A total

score was calculated by summing all 38 items and

comparing them using a repeated measures t-test for

the entire group and by location.

THE SHORT-TERM BENEFITS OF EDUCATIONAL ROBOTICS WHEN PAIRED WITH GEOSPATIAL

TECHNOLOGIES IN INFORMAL LEARNING ENVIRONMENTS

395

Table 3: Attitudinal paired samples test.

3 RESULTS

3.1 Learning

Overall there was a significant increase from the

pretest (M = 15.63, SD = 4.52) to the posttest scores

(M = 20.12, SD = 5.60, t (136) = -13.71), p < .001)

for the combined groups. These results suggest that

the 4-H robotics and the geospatial summer camp

program is a promising approach for supporting

STEM-related learning. Results suggest that, overall,

youth had significant increases in scores. Each

location except for Omaha North had a significant

mean increase (p < .001) from pre to posttest. See

Table 2.

To get a better understanding of how individual

sites scored on the test; the mean scores are

separated by location. See Figure 1. The sites are

listed in chronological order with the Omaha North

camp first and ending with the Grand Island camp

six weeks later.

3.2 Attitude

Similar to the knowledge instrument participants

scored significantly higher on the posttest

(M=155.91, SD = 20.20) than on the pretest

(M=147.52, SD = 22.03, t (133) = -5.09), p < .001)

indicating the 4-H robotics and GPS/GIS summer

camps have a positive short-term effect on attitudes

towards STEM topics. While all sites excluding the

Omaha North had pre-post increases in attitudinal

means, the t-results were not as significant as those

from the content test. In addition, three sites did not

have significant increase in scores. See table 3.

Figure 1: Pre and posttest mean scores by location for the

content test.

4 DISCUSSIONS AND

CONCLUSIONS

The significant increase in student scores on the

learning assessment provides evidence for the use of

robotics and GIS/GPS technologies as a means to

promote STEM learning. With one exception, all the

sites produced short-term gains from pre to posttest.

Location

Paired Differences

Pre Mean

Post Mean

Mean

Std. Deviation

Sig. (2-tailed)

Omaha North

152.93

152.87

.067

22.72

.011

.991

Omaha South

148.09

153.45

-5.36

9.39

-1.89

.088

Lincoln

149.39

156.13

-6.75

14.95

-3.69

.001

Ord

148.11

161.67

-13.56

26.81

-1.52

.168

Chadron

138.13

151.06

-12.94

22.98

-2.25

.040

Grand Island

143.31

161.13

-17.81

23.89

-2.98

.009

CSEDU 2010 - 2nd International Conference on Computer Supported Education

396

The Omaha North site did not show significant

improvement from pre to posttest scores. One

plausible explanation for the lack of improvement at

that site is that chronologically it was the first camp

run by the project staff. Therefore, activities and

presentation methods were still relatively new, and

were still evaluated and refined. This can be

supported by the apparent increase of the mean

paired difference between post and pretest that

occurred later in the program. Another difference

with the Omaha North site is that it had a lower

mean score on the pretest (M=10.80, SD = 3.22)

compared to other sites. The lower pretest score

may indicate that this particular group of youth did

not have as much initial experience and therefore,

prior knowledge of robotics and geospatial concepts

as other groups, perhaps suggesting that at least a

minimal level of initial understanding of these topics

is needed for students to be fully successful with this

level of activities.

Documenting the positive impacts of robotics

and GPS/GIS activities on student’s attitudes has

been a struggle in past research (Nugent, Barker, &

Grandgenett (2008). Prior to this study the project

team piloted two other existing attitude instruments

(Scientific Attitude Inventory, Moore & Foy. 1997;

Pell & Jarvis, 2001) with nonsignificant pre to post

comparisons. Past results suggest that youth have a

difficult time in making the connection between

STEM concepts and Robotics and GPS/GIS

activities. When robotics and GPS/GIS are

embedded into a natural experiential learning

environment, as opposed to the more traditional

direct instruction, students may become excited

about robotic and GPS/GIS, but not recognize that

STEM learning is actually being integrated into the

activities. Results have led to curricular revisions,

including specific instruction on how robotics

activities relate to science, engineering, math and

technology and the creation of a new measurement

tool.

The results of this study indicate that our

attitudinal measurement instrument can detect short-

term attitudinal changes towards STEM. More

research is needed to examine each of the eight

constructs and to assess various trends and the

potential interactions of these constructs with

participant demographics.

ACKNOWLEDGEMENTS

This material is based upon work support by the Na-

tional Science Foundation under Grant No. ESI-

0624591 and DRL-0833403

REFERENCES

Alsop, S. & Watts, M. (2003). Science education and

affect. International Journal of Science Education. 25

(9), 1043-1047.

Barker, B. & Ansorge, J. (2007). Robotics as means to

increase achievement scores in an informal learning

environment. Journal of Research on Technology

Education 39(3), 229-243.

Barker, B., Nugent, G., Grandgenett, N., Hampton, A.

(2008) Examining 4-H robotics in the learning of

science, engineering and technology topics and the

related student attitudes. Journal of Youth

Development: Bridging Research and Practice Volume

2(3). On-line:

http://www.nae4ha.org/directory/jyd/jyd_article.aspx?i

d=f5a34e58-1cd3-4994-981d-b81fa406cd74

Halpern, D., Aronson, J., Reimer, N., Simpkins, S., Star,

J., Wentzel, K. (2007). Encouraging Girls in Math and

Science. Washington DC: National Center for

Education Research.

Koballa, T. & Glynn, S. (2007). Attitudinal and

motivational constructs in science learning. In S. Abell

and N. Lederman (Eds,) Handbook of Research in

Science Education (pp. 75-102). Mahwah, New

Jersey: Erlbaum.

Lester, B. (2007). Robots’ allure: can it remedy what ails

computer science? Science, 318, 1086-1087.

McWillimas, H. & Rooney, P. (1997). Mapping our city:

Learning to use spatial data in the middle school

science classroom. Paper presented at the annual

meeting of the American Educational Research

Association. Chicago, IL.

Moore, R. & Foy, R. (1997). The scientific attitude

inventory: A revision (SAI II). Journal of Research in

Science Teaching, 34(4), 327-336.

Moore, R. & Sutman, F.X. (1970). The development, field

test, and validation of an inventory of scientific

attitudes. Journal of Research in Science Teaching, 7,

85-94.

Nourbakhsh, I., Crowley, K., Bhave, A., Hamner, E.,

Hsium, T., Perez-Bergquist, A., Richards, S. &

Wilkinson, K. (2005). The robotic autonomy mobile

robots course: Robot design, curriculum design, and

educational assessment. Autonomous Robots, 18(1),

103-127.

Nugent, G., Barker, B., & Grandgenett, N. (2008). The

effect of 4-H robotics and geospatial technologies on

science, technology, engineering, and mathematics

learning and attitudes. Proceedings of the Ed Media

Conference, Vienna, Austria, July 4, 2008.

Pell, T. & Jarvis (2001). Developing attitude to science

scales for use with children of ages from five to eleven

years. International Journal in Science Education,

23(8), 847-862.

THE SHORT-TERM BENEFITS OF EDUCATIONAL ROBOTICS WHEN PAIRED WITH GEOSPATIAL

TECHNOLOGIES IN INFORMAL LEARNING ENVIRONMENTS

397

Pintrich, P.,Smith, D., Garcia, T., & McKeachie, W.

(1991). A Manual for the Use of the Motivated

Strategies for Learning Questionnaire (MSLQ). Ann

Arbor, MI: University of Michigan.

Rogers, C., & Portsmore, M. (2004). Bringing engineering

to elementary school. Journal of STEM Education,

5(3&4), 17-28.

Wanner, S.& Kerski, J. (1999). The effectiveness of GIS

in high school education. Paper presented at the ESRI

1999 users conference. Available at:

http://gis.esri.com/library/userconf/proc99/proceed/pa

pers/pap203/p203.htm

CSEDU 2010 - 2nd International Conference on Computer Supported Education

398