Round Table

Distance learning profile instrument: Predicting on-line course achievement

 

Roy Clariana & Les Mohler, Penn State, Great Valley Graduate School, March 2000

 

For AECT Annual Convention in Denver, CO in October 2000    http://www.aect.org/

 

 

 

Abstract

This round table shares a 20-item distance learning profile instrument (DLP) that predicts student grades (success) in distance learning.  This paper describes how the instrument was developed and provides correlations between DLP self-report items and online course achievement for several courses.  Factor analysis of the items revealed either 3 factors that include in order of importance: likely student engagement with the course (includes course quality), student ability to work independently, and student competitiveness.

 

Initial DLP Development

In Phase 1, undergraduate students (N=35) enrolled in 5 separate online full-credit courses completed 85 learning preference items adapted from four separate self-report instruments: Grasha-Reichmann Student Learning Style Scale, the Pennsylvania State University Student Rating of Teaching Effectiveness (SRTE), the Rotter Scale, and the Productivity Environmental Preference Survey (PEP).  These responses were compared by correlation to course achievement, and 20 items were selected that were most related to course achievement.  Next the responses to these 20-items were examined by factor analysis; and then the relationship of these identified factor groupings to course achievement was examined by multiple-regression.  Five factors were identified and were described based on the items, these include: Active Engagement (multiple r = 0.82, 67%), Independent Learner (r = 0.64, 41%), Course Quality (r = 0.41, 17%), Competitive (r = 0.39, 15%), and other (r = 0.21, 4%).  These factors are listed in Table 1.  The factors are listed in the left column along with the factor correlation with course achievement.  The self-report items in each factor are listed in the right column in order of relationship to the factor, a negative is placed beside those factors that exhibited negative correlation to the factor.  Note that sometimes, the items are stated in “negative”.  Generally, "course quality" is part of "student engagement" and "other" is part of "independent learner".

 

 

Table 1.  DLP Factor categories identified in Phase 1.

Factors

Self-report items

Active Engagement

(r=0.82, 67%)

1. Course assignments are interesting.

7. The course assignments are appropriate.

- 4. Many of the course activities seem useless.

- 10. The course is boring.

13. I work harder than others to stand out from the crowd.

16. This course actively engages me.

20.This course "turns me on"!

Independent Learner

(r=0.64, 41%)

2. I learn best without supervision.

5. I am a self-starter.

- 8. I usually prepare for exams well in advance.

- 11. I prefer constant feedback from the teacher.

14. I don't care how others are doing on assignments.

Course Quality

(r=0.41, 17%)

17. Overall, I consider this to be a high quality course.

3. I prefer tough courses that really challenge me.

Competitive

(r=0.39, 15%)

18. I am usually competitive.

9. I make sure that other students get my viewpoint.

6. I always try to out perform other students.

other

(r=0.21, 4%)

19. I prefer to do assignments my way.

- 12. My views contribute little to the quality of a course.

15. I work best under a deadline.

 

The factors active engagement and independence were highly correlated with course achievement. Students that are actively engaged are likely taking maximum advantage of the course resources, while independent learners probably are initiating there own learning; alternately students not engaged and not independent neither initiate any learning or additional experiences and so gain little advantage from the course information and resources.  These may just be waiting around for someone to tell them what to do next.

With these reasonable findings, the 20 self-report items identified in Phase 1 were given to a second group of students.  In Phase 2, graduate students (N=15) in a management course delivered online completed the 20 item DLP before taking the course in order to determine whether the DLP instrument could predict future course success.  As in Phase 1, the responses to these 20-items were examined by factor analysis and the factors were related to course achievement by multiple-regression.  Again five factor item groupings were obtained. These items included in the factors are similar to those observed in Phase 1, though the exact items in each factor were slightly different.  These include: Active Engagement (multiple r = 0.81, 65%), Independent Learner (r = 0.28, 8%), Course Quality (r = 0.42, 18%), Competitive (r = 0.40, 16%), and other (r = 0.58, 34%).

 

Table 2.  DLP Factor categories identified in Phase 2.

Cluster

Questions

Active Engagement

(r=0.81, 65%)

-4. Many of the course activities seem useless.

1. Course assignments are interesting.

16. This course actively engages me.

17. Overall, I consider this to be a high quality course.

7. The course assignments are appropriate.

-10. The course is boring.

20. This course "turns me on"!

Independent Learner

(r=0.28, 8%)

14. I don't care how others are doing on assignments.

15. I work best under a deadline.

11. I prefer constant feedback from the teacher.

5. I am a self-starter.

Course Quality

(r=0.42, 18%)

3. I prefer tough courses that really challenge me.

18. I am usually competitive.

-12. My views contribute little to the quality of a course.

Competitive

(r=0.40, 16%)

13. I work harder than others to stand out from the crowd.

-8. I usually prepare for exams well in advance.

6. I always try to out perform other students.

other

(r=0.58, 34%)

2. I learn best without supervision.

9. I make sure that other students get my viewpoint.

19. I prefer to do assignments my way.

 

            The same general factors were described as in Phase 1, but the items sorted out slightly differently.  The two primary factors identified in both Phase 1 and 2, Active Engagement and Independent Learner, had nearly the same item groupings in Phases 1 and 2; but the remaining three factors, Course Quality, Competitive, and other, were very different.

            In both cases, DLP items and factors explained a large amount of variance in course achievement.  The factor groupings were relatively consistent for both groups.  The actual factor descriptions and names will require revision, but this seems to be a good start towards a self-report instrument that may predict individual success in online courses.

            Obviously, this DLP instrument is preliminary; but we would like the opportunity to share it with colleagues in a round table session (see Table 3).

Table 3. Distance Learning Profiler (DLP)

 

 

Based on your knowledge of the course so far:

Disagree - - - - - - - - - - Agree

1    2    3    4    5    6     7

1. Course assignments are interesting.

1    2    3    4    5    6     7

2. I learn best without supervision.

1    2    3    4    5    6     7

3. I prefer tough courses that really challenge me.

1    2    3    4    5    6     7

4.  Many of the course activities seem useless.

1    2    3    4    5    6     7

5. I am a self-starter.

1    2    3    4    5    6     7

6. I always try to out perform other students.

1    2    3    4    5    6     7

7.  The course assignments are appropriate.

1    2    3    4    5    6     7

8. I usually prepare for exams well in advance.

1    2    3    4    5    6     7

9. I make sure that other students get my viewpoint.

1    2    3    4    5    6     7

10. The course is boring.

1    2    3    4    5    6     7

11. I prefer constant feedback from the teacher.

1    2    3    4    5    6     7

12. My views contribute little to the quality of a course.

1    2    3    4    5    6     7

13. I work harder than others to stand out from the crowd.

1    2    3    4    5    6     7

14. I don't care how others are doing on assignments.

1    2    3    4    5    6     7

15. I work best under a deadline.

1    2    3    4    5    6     7

16. This course actively engages me.

1    2    3    4    5    6     7

17. Overall, I consider this to be a high quality course.

1    2    3    4    5    6     7

18. I am usually competitive.

1    2    3    4    5    6     7

19. I prefer to do assignments my way.

1    2    3    4    5    6     7

20. This course "turns me on"!