Research Justification of PSE by Canelos, J.

Canelos,J. (1987) Research Justification and Implementation of the Program of Systematic Evaluation (PSE) in Dwyer, F.(Editor). Enhancing Visualized Instruction-Recommendations for Practitioners, Pa: Learning Services, pp. 1-3.

A series of experimental studies beginning in 1965 and employing similar instructional materials (Dwyer, 1972) has come to be known as the Program of Systematic Evaluation (PSE). The PSE has generated more than 150 published and descriptive research studies involving approximately 35,000 subjects at the college and high school levels. Generic to the studies are: (1) an instructional script containing approximately 2000 words describing the parts of the heart, their locations, and their simultaneous functions during the diastolic and systolic phases, (2) nine visualized sequences containing different degrees of realistic detail of the heart, and (3) four criterion measures used to evaluate different types of educational objectives. Collectively, the instructional script, visualized sequences, and criterion measures have been designated as the Experimental Instructional Materials (ElM). These materials, systematically manipulated in relation to other independent variables, have facilitated systematic explorations necessary to add to the body of knowledge regarding the design, development, and implementation of visualized learning environments,which have a high degree of predictability in facilitating the achievement of specific educational objectives.

Studies conducted using the PSE have been published in experimental Journals, applied Journals, conference proceedings, and as textbook chapters. Researchers involved in PSE have included graduate students at the thesis and dissertation levels, researchers in academic areas such as education, medicine, and engineering and researchers industry, business, and military environments. The results of studies conducted in the PSE have contributed significantly to the knowledge base of instructional systems and educational communications research. Additionally, research results obtained in the PSE have yielded important findings regarding psychological factors contributing to learning such as motivation, memory, cognitive learning strategies, and testing/evaluation methods.

back to top

An examination of the Experimental Instructional Materials (ElM) reveals that this instructional package not only facilitates learning about the human heart but represents generic stimulus and criterion materials that have been found to be valid across a wide range of experimental and control conditions. Consequently, these materials have been very effective in evaluating specific internal and external conditions affecting learning, learner behavior, instructional design strategies, and presentation methodologies.

Consider this scenario: a researcher has an idea, let's say, to evaluate the effects of encoding specificity when instruction is presented by visualized instruction versus non-visualized instruction and visualized testing versus non-visualized testing. Where does this researcher begin? If no instructional or stimulus materials are available, they must be designed, developed, tested, and validated. Similarly, any tests or criterion materials must be checked not only for validity and reliability but also for intellectual skill level, objectives evaluated, difficulty level, and so forth.

Therefore,the problems confronting many researchers is not so much designing unique studies but finding criterion and stimulus materials that are valid and that operationally fit their hypotheses. Such materials can usually be developed, and in many cases this is the only solution. However, this involves time and considerable expense. In addition, many experimental studies use simplistic and totally unrealistic learning and testing materials, making generalization to the classroom and/ or training environment difficult at best. The ElM represent a typical academic instructional unit focusing on the physiology and functions of the heart and represent types of mediated instruction which learners have previously experienced.

The ElM are representative of those materials typically used in conventional instruction and testing environments, thereby making experimental results using these materials generalizable to traditional classroom and testing conditions. The fact that the instructional materials deal with the human heart is irrelevant unless the subject pool has significant prior knowledge of heart physiology. What is relevant is that the instructional sequence and presentation, which makes up the basic instructional program, can be described in specific operational terms, contingent upon how the program is presented (i.e., visual and verbal slides, interactive computer based instruction, verbal programmed booklets, television, etc.). Additionally, the criterion, or testing, materials on the heart content can be defined in terms of specific intellectual tasks or objectives (i.e., list learning, spatial learning, concept learning, and problem solving). Finally, the ElM have been validated, and the tests have been shown to be highly reliable. They are generalizable to the classroom learning environment, eliminating the need to: (1) develop new materials, (2) validate and check those materials for reliability, and (3) evaluate new materials for generalizability.

back to top

Studies in the PSE utilizing the ElM have evaluated a wide range of cognitive learning factors and presentation methods. Conclusions derived from these studies help instructional systems professionals and those involved with the uses of educational technology to design better instructional, training, and evaluation programs within a given curriculum and also to apply appropriate educational technologies. Presented in the text are a number of conclusions from these studies which can be applied to the design of learning environments.

A 1986 AECT Research and Theory Division Proceedings paper by Canelos, Baker, Taylor, Belland, and Dwyer describes an experiment using a unique application of computer based instruction. They looked at the effects of embedded cueing strategies, program pace strategies, and effects on five types of intellectual skills, and found a significant effect favoring the use of program embedded learning strategies when teaching with CBL

Parkhurst and Dwyer, in an experimental study published in the Journal of Instructional Psychology, 1983, examined the effect of I.Q. on learning from simple versus more complex visuals. Their results yielded an interaction indicating that those with higher I.Q. learn better from more complex visual displays. A 1985 study, published in the International Journal of Instructional Media (Jennings and Dwyer), evaluated visual cueing strategies when instruction was presented by print materials. They found that more elaborate visual cues tended to be more effective. Joseph and Dwyer (1984) in the Journal of Experimental Education examined the cognitive variable of prior knowledge and the instructional factors of external versus self-paced instruction and types of visuals. They found that visuals with varied degrees of realistic detail can be used to reduce differences in the performance of learners with different levels of prior knowledge. Encoding specificity as a cognitive factor was examined by Szabo, Dwyer, De M:lo f'' in 1981 and was reported in Educational Communications Technology Journal. Their findings are consistent with the encoding specificity hypothesis indicating that combining visualized instruction and visualized testing improves performance.

Nesbit examined the relationship of the psychological variable of I.Q., the physiological variable of eye fixation using an oculometer system, and the complexity of instructional visualization. The results of his research in the 1981 Educational Communications and Technology Journal indicated that learners with higher I.Q. fixate more on complex instructional visuals, and learners with lower I.Q. have fewer fixations as visual complexity increases. A study by Mary Ann Chezik (1987) systematically examined the effects of review and practice activities when instruction was presented by print materials. A comprehensive " literature review dealing with the effects of color and color coding was presented by Dwyer and Lamberski in 1982-83 in International Journal of Instructional Media and describes the results of 185 research studies.

back to top

The instructional strategy of note-taking was examined by Canelos, Dwyer, Taylor, and Nichols in a study reported in 1984 in the Journal of Instructional Psychology. They found that learners can be trained to use effective note-taking methods and that a method providing a pre-structure to the lesson is more effective than most learner-evolved note-taking methods. A 1984 study, reported in the Journal of Visual and Verbal Lanua!n by Dwyer and De Melo, examined the encoding specificity hypothesis and its relationship to visualized instruction and visualized testing. They found general support for the encoding specificity hypothesis within the context of typical school learning and testing materials. In an article appearing in Programmed Learnin and Educational Technology, Dwyer and Parkhurst (1984) examined the instructional variables on visual complexity and program pace and the cognitive variable reading comprehension. They found that learners with lower reading comprehension learned better with simpler visuals, while learners with higher reading comprehension perform better with more complex instructional visuals.

Dwyer's 1982 article in the Journal of Instructional Psychology evaluated the use of attention-directing strategies to improve learning during instructional television presentations. In general, he found the pre-program question cueing strategy to be more effective than, or at least as effective as, motion and static arrow cues in facilitating students' achievement of different types of instructional objectives. A 1982 AECT Research and Theory Division Proceedings paper by Roberts examined the validity of the ElM for teaching specific instructional content to naive learners and found them to be effective in communicating different types of information. Visual literacy is emphasized in conjunction with PSE by Dwyer in 1985 in an article in the Journal of Visual Verbal Languaging. He described a number of the variables considered in the PSE and related them to current issues in the visual literacy movement. A study by Lipsky, appearing in 1984 in Focus on Learning, evaluated different instructional television teaching methods. He found that an active viewing method, in which the television program is interrupted by questions, is more effective for learning than conventional television viewing methods.

An experimental study by Lamberski and Dwyer considered color coding techniques as an instructional strategy. They found that color coding improved attention and learner motivation and the formation of structure in memory. A study in 1983 by Canelos examined imagery and imagery learning strategies. This study, published in Journal of Experimental Education, evaluated three types of imagery learning strategies and found that learners can be trained to use imagery as a memory strategy. , It was also determined that chunking of information can be a beneficial feature when using imagery learning strategies. McBride and Dwyer, in 1985 in the Journal of Experimental Education, considered the organization of instructional materials and post-questions versus instructional material chunking methods and found the chunked method to be more effective and efficient for learning.

back to top

A 1984 study by Wise, appearing in the International Journal of Instructional Media, evaluated the psychological variables of field-dependence and field-independence and how students possessing these two cognitive styles learn from instructional visuals which vary significantly in terms of visual complexity. Two studies by Canelos and Taylor and Canelos, Taylor and Altschuld further examined the use of imagery learning strategies and visualized instruction. A 1981 study in the Journal of Experimental Education and a 1982 study in Educational Communications Technology Journal reported findings that an imagery strategy involving networking, or information chunking, was generally more effective for processing new information. These two studies also examined the use of imagery strategies when learners dealt with different intellectual skill levels.

Acevedo and Lamberski used the ElM to evaluate how bilingual learners interact with different types of visual complexity. Their research was published in the 1980 AECT Research and Theory Division Proceedings. The cognitive variable of encoding specificity was examined in 1985 and reported in the International Journal of Instructional Media by Canelos, Taylor, and Dwyer. Their results generally supported past research, indicating that if visuals are a significant part of instruction, visual cues should also be a significant part of the testing situation.

A study, appearing in a special issue of Educational Communications and Technology Journal in 1985 on uses of computers in instruction, conducted by Belland, Taylor, Canelos, Dwyer, and Baker, yielded some surprising conclusions. They examined different pacing strategies for the design of microcomputer-based instructional programs and found externally paced methods to be more effective for learning than self-paced methods. Also examined was intellectual achievement across five tasks that increased in difficulty: list learning, spatial learning/cued recall, simple concept learning, complex concept learning, and spatial problem-solving. A study by Dwyer and Dwyer, published in the Journal of Visual Verbal Languaging in 1985, evaluated rehearsal activities, their effects upon learning from different types of visuals, and delayed retention of cognitive information acquisition.

Dealing with the psychological factor of I.Q., Berry and Dwyer found a complex relationship between varied I.Q. levels and color realism in instructional visuals. Their results were presented in a 1982 issue of Perceptual and Motor Skills. An interesting study by Olsen in 1985 was published in the AECT Research and Theory Division Proceedings. Olsen evaluated the strategy of rate modified speech for teaching technical content and its effects upon learning from verbally and visually augmented materials. A study by Carol Dwyer, appearing in Journal of Experimental Education, 1985-86, examined the variables of encoding specificity, transfer appropriate processing, and rehearsal strategy. This study provided a comprehensive literature review of encoding specificity and transfer appropriate processing. Her results indicated that visual rehearsal strategies were most effective. The cognitive style variable of dogmatism was evaluated by Berry and Dwyer in a 1982 study appearing in the International Journal of Instructional Media. They found that high and low dogmatic learners interacted with visual complexity differently.

back to top

Carol Dwyer, in a series of studies (1985, 1985-1986, 1986) which examined the instructional effect of visual rehearsal, rehearsal time, and visual testing on students' level of information acquisition and immediate and delayed retention, found that: (a) all types of rehearsal strategies are not equally effective in facilitating student achievement of different educational objectives, (b) visual testing is a valid testing strategy for assessing students' learning from visualized instruction, and (c) students who are afforded quality interaction opportunities (practice) spend more time learning and achieve significantly better on tests measuring specific educational objectives. Two in-press studies by Buckley and Dwyer and Barton and Dwyer found that the amount of rehearsal activity significantly influenced the performance of students identified as external locus of control and that the addition of audio redundancy to an instructional sequence has a significant negative effect on the achievement of learners identified as being in the low I.Q. level.

In these recent research studies a wide range of variables dealing with instructional design and human learning have been examined using the ElM as the vehicle for conducting the research. These generic materials allowed for the examination of independent variables such as: cognitive style, i intellectual skill level, intelligence, prerequisite skills, imagery learning, memory strategies, encoding specificity, transfer appropriate processing, and others. In addition, some of the instructional design variables included: computer-based instruction, note-taking methods, instructional television, testing strategies, visual design, instructional pacing strategies, feedback strategies, and cueing methods. Hopefully, efforts will be made to continue to implement the results of the present studies at the practical level in educational and training environments and that these studies will stimulate further research in related areas.

back to top