Using PATHFINDER KNOT aNALYTIC TOOLS FOR COMPARING
 AND COMBINING CONCEPT MAPS

Roy Clariana, Ravinder Koul, and Kristen Albright

Abstract. This investigation describes a Pathfinder software-based method for quantitatively comparing students’ concept maps to each other, to an expert map, and also for combing several concept maps into one map that represents collective ‘group’ knowledge. Participants were 16 graduate-level students enrolled in a curriculum course. Participants worked in dyads to convert a 300-word text passage on the heart and circulatory system into a concept map of this content. The 8 resulting concept maps were converted to proposition-as-text files, and these text files were converted into 8 proximity files by free ALA-Reader software, and then the proximity files were analyzed using Pathfinder KNOT software. The percent of overlap of the 8 concept maps are provided in a table below. In addition, KNOT software was used to average the 8 maps together to form one network representation (a concept map like visual).

 

Category/Categoría: Poster

1         Introduction

Concept maps are a valuable tool for representing knowledge (Novak, n.d.; Novak & Gowin, 1984). After working hard on a concept map, it is natural that a student would want to compare their concept map to another student’s map, or perhaps to the teacher’s map in order to compare similarities and differences.  CmapTools (v.4x) has a Compare-to-Cmap feature in the Tools menu that allows students to do this (e.g., see: http://cmap.ihmc.us/Support/Help/Compare.htm).

Along the same lines, teachers are interested in comparing students’ concept maps to each other and to an expert referent to inform both instruction and assessment. Though student’s can use the Cmap Compare-to-Cmap feature on the fly, the feature is cumbersome for the teacher to use for making multiple comparisons among students.  For these reasons, we are interested in alternate easy-to-use approaches for quantitively comparing multiple concept maps. This poster describes an existing software tool called Pathfinder Knowledge Network Organizing Tool (KNOT; Schvaneveldt, 1990) that has extensive analyses capability including simultaneous comparisons between multiple concept maps as well as the capability of combining multiple concept maps into a single group network representation (Pathfinder KNOT software is available online at: http://interlinkinc.net/KNOT.html).

2         Using Pathfinder KNOT to Analyze Concept Maps

To analyze concept maps using KNOT software, first, students completed concept maps must be saved as a proposition text file using the Export-Cmap-As feature in the CmapTools File menu, and then the teacher can easily convert these text files into KNOT proximity files using ALA-Reader software (see Figure 1). Finally, the proximity files created by ALA-Reader can be opened and analyzed by KNOT software.

 

 

Figure 1.  A mock concept map (left), the Propositions-as-text file of this map (center) generated using the Export-Cmap-As feature, and the proximity array (right) generated by ALA-Reader from this text file.

ALA-Reader software (available at no cost, see: http://www.personal.psu.edu/rbc4/score.htm) was designed to convert text passages into proximity files, thus allowing teachers to use KNOT to compare students’ essays to each other and to an expert referent (Clariana & Koul, 2004). The Mark S feature in ALA-Reader is also able to convert Cmap proposition-as-text files directly into KNOT proximity files, given a list of common terms across all maps.

2.1          Source of the Concepts Maps Used for Analysis

In order to try this analysis approach, we required multiple similar concept maps on the same topic. A group of 16 graduate-level students enrolled in one of our curriculum courses assisted us in this investigation. We decided to parallel an approach used by Lomask, Baron, Greig, and Harrison (1992) where they converted students’ essays of science content knowledge into concept maps, and then scored the concept maps using a quantitative rubric. Working in dyads, our graduate students converted a student-written 300-word text passage on the heart and circulatory system into concept maps of this content. We provided them with a list of important terms and an expert’s concept map of this topic ahead of time. Note that the members in Dyads 1, 4, 5, 7 and 8 were non-science majors while one of the members in each of the Dyads 2, 3, and 6 were science majors. Our graduate students were instructed to make a concept map that accurately represented the text passage, not their own understanding of the topic. We used this approach in order to generate concept maps for comparison purposes that are as similar as humanly possible. However, the eight Dyads produced quite different numbers of propositions. The non-science majors generally included fewer propositions in their maps, except for Dyad 1 (Dyad 1 had 42, Dyad 4 had 11, Dyad 5 had 19, Dyad 7 had 15, and Dyad 8 had 15), while the science major dyads had relatively more propositions in their maps (Dyad 2 had 51, Dyad 3 had 29, and Dyad 6 had 34). The expert map that they were given ahead of time had 16 propositions (see Figure 2).

The eight resulting concept maps (one map from each dyad) were converted to proposition-as-text files and these text files were converted into eight proximity files using ALA-Reader software (refer back to Figure 1) using a list of 26 important terms (including synonyms and metonyms). In addition, the student-written 300-word text passage was also converted by ALA-Reader into a proximity file.

 

 

Figure 2.  An expert concept map of this topic (note that the ALA-Reader proposition-as-text format requires
the addition in the top left corner of Concept boxes for the Title, the expert’s name, and a space).

2.2          KNOT Analysis Results

The proximity files were converted to network files and then analyzed by KNOT software. The software used the list of 26 important terms, and in this analysis, the values for Minkowski’s r was infinity and q was equal to 25 (e.g., n - 1). In the analysis, each dyad’s concept map network was compared to every other dyad and to the 300-word text. In addition, KNOT software calculated an ‘average’ proximity file that is the combination of all 8 dyads, thus providing a measure of collective group knowledge. The group average was also compared to all of the dyads and to the 300-word text network. A measure of percent of agreement was calculated by dividing the number of links in common by the average number of links in the network pair. The results of this analysis are displayed in Table 1.

 

 

Non-Science Majors

 

Science Majors

 

D1

D4

D5

D7

D8

 

D2

D3

D6

Dyad 1

 --

0.08

0.13

0.07

0.07

 

0.32

0.28

0.32

Dyad 4

0.08

 --

0.07

0.23

0.08

 

0.03

0.00

0.04

Dyad 5

0.13

0.07

 --

0.06

0.00

 

0.51

0.50

0.45

Dyad 7

0.07

0.23

0.06

 --

0.00

 

0.12

0.14

0.12

Dyad 8

0.07

0.08

0.00

0.00

 --

 

0.06

0.09

0.08

Dyad 2

0.32

0.03

0.51

0.12

0.06

 

 --

0.70

0.68

Dyad 3

0.28

0.00

0.50

0.14

0.09

 

0.70

 --

0.89

Dyad 6

0.32

0.04

0.45

0.12

0.08

 

0.68

0.89

 --

average

0.27

0.19

0.34

0.17

0.28

 

0.57

0.43

0.49

300-word text

0.23

0.19

0.33

0.16

0.27

 

0.53

0.39

0.45

Table 1:   The average percent of agreement for each pair of concept map networks sorted by major.

 

We expected that the percent of agreement between the pairs of dyad concept map networks would be quite high, since they were all derived from the same 300-word text passage using the same list of terms. However, this was not the case (see Table 1). The science major dyads were fairly similar (68%, 70%, and 89%), with the largest percent agreement between Dyads 3 and 6 (89%). For the non-science dyads, the percent agreement values were considerably lower. The percent agreement between each dyad and the 300-word text provides another objective measure of the quality of the dyad maps (in terms of concept links only). Dyad 2 had the largest percent agreement (e.g., 53%) with the 300-word text network.

 

 

Figure 3.  The relationship between the number of propositions in the original dyad concept maps
and the percent agreement with the 300-word text passage network.

 

Further analysis indicates that the quality of each dyad’s concept map network is strongly related to the number of propositions in their concept map (see Figure 3). The correlation between the number of propositions in the original dyad concept maps and the percent agreement with the 300-word text passage network is r = .94, the exception is Dyad 1, that had 42 propositions in the concept map but only 23% agreement with the 300-word text passage. This data suggests that the dyads were fairly accurate at translating text propositions into concept map propositions, but that some dyads were far more thorough than others. Apparently, prioritizing the importance of information to include in a concept map is a critical task in concept map creation.

One other measure that was calculated is group average. Group average is calculated by KNOT software by averaging together the cell values of all eight of the proximity array files, and then generating a network representation of the total group. The resulting group average network obtained a 62% agreement with the 300-word text network. In addition, the science major dyads dominated the group average network structure (57%, 43%, and 49%; see the bottom right side of Table 1), probably because they are more alike and contain more propositions. Specifically, when KNOT averages propositions across files, common propositions survive while idiosyncratic propositions drop out.

3         Summary

In their study on the impact of subject matter (content) knowledge on teaching, Hashweh (1987) found that teachers could list many methods and ideas for teaching concepts in their specialty areas but showed misconceptions and misunderstandings about content knowledge in the field that is not their major field.  In our study, the quality of the concept map network appears to be related to the participants’ domain knowledge of science:  Dyads 2, 3 and 6, with one science major in each dyad, were more accurate in translating text propositions into concept map propositions; Dyads 1, 4, 5, 7, and 8 with no science major, were less accurate in translating text propositions into concept map propositions.  While not conclusive, teachers may be less accurate when converting an essay into a concept map if the content of the essay is outside their area of specialty. Our original intent was to describe an easy-to-use approach for quantitatively comparing concept maps. We used our graduate students, a sample of convenience, to convert the 300-word text into concept maps without regard to their background. However, during data analysis, we noted the substantial differences between dyads, and realized the likely importance of dyad prior knowledge in concept map formation, even when their task was only to convert a text to a concept map (specifically directed NOT to use their own knowledge). This finding seems critical in concept map creation.

4         Acknowledgements

This Research Project was provided with graduate student support time by the Education Department at Penn State, Great Valley Campus.

References

Clariana, R.B., & Koul, R.  (2004). A computer-based approach for translating text into concept map-like representations. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17, pp.131-134.  Downloaded April 9, 2006 from http://cmc.ihmc.us/papers/cmc2004-045.pdf

Hashweh, M. Z. (1987).  Effects of subject matter knowledge in the teaching of biology and physics.  Teaching and Teacher Education, 3, 109-120.

Lomask, M., Baron, J., Greig, J., & Harrison, C. (March, 1992). ConnMap: Connecticut's use of concept mapping to assess the structure of students' knowledge of science. A symposium presented at the annual meeting of the National Association for Research in Science Teaching, Cambridge, MA.

Novak, J. D. (n.d.). The theory underlying concept maps and how to construct them. Downloaded April 9, 2006 from http://Cmap.coginst.uwf.edu/info/printer.html

Novak, J. D. & Gowin, D. B. (1984) Learning How to Learn. Cambridge: Cambridge University Press.

Schvaneveldt, R. W. (Editor) (1990). Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex

ding:0in 5.4pt 0in 5.4pt;