Using PATHFINDER KNOT
aNALYTIC TOOLS FOR COMPARING
AND COMBINING CONCEPT MAPS
Roy
Clariana, Ravinder Koul, and Kristen Albright
Abstract. This investigation describes a Pathfinder software-based method for quantitatively comparing students’ concept maps to each other, to an expert map, and also for combing several concept maps into one map that represents collective ‘group’ knowledge. Participants were 16 graduate-level students enrolled in a curriculum course. Participants worked in dyads to convert a 300-word text passage on the heart and circulatory system into a concept map of this content. The 8 resulting concept maps were converted to proposition-as-text files, and these text files were converted into 8 proximity files by free ALA-Reader software, and then the proximity files were analyzed using Pathfinder KNOT software. The percent of overlap of the 8 concept maps are provided in a table below. In addition, KNOT software was used to average the 8 maps together to form one network representation (a concept map like visual).
Category/Categoría: Poster
1 Introduction
Concept maps are a valuable tool for representing knowledge (Novak, n.d.;
Novak & Gowin, 1984). After working hard on
a concept map, it is natural that a student would want to compare their concept
map to another student’s map, or perhaps to the teacher’s map in order to
compare similarities and differences. CmapTools (v.4x) has a Compare-to-Cmap feature
in the Tools menu that allows students to do this (e.g., see: http://cmap.ihmc.us/Support/Help/Compare.htm).
Along the same lines, teachers are interested in
comparing students’ concept maps to each other and to an expert referent to
inform both instruction and assessment. Though student’s can use the Cmap Compare-to-Cmap feature on the fly,
the feature is cumbersome for the teacher to use for making multiple
comparisons among students. For these
reasons, we are interested in alternate easy-to-use approaches for quantitively
comparing multiple concept maps. This poster describes an existing software
tool called Pathfinder Knowledge Network
Organizing Tool (KNOT; Schvaneveldt,
1990) that has extensive analyses capability including simultaneous comparisons
between multiple concept maps as well as the capability of combining multiple
concept maps into a single group network representation (Pathfinder KNOT software
is available online at: http://interlinkinc.net/KNOT.html).
2
Using
Pathfinder KNOT to Analyze Concept
Maps
To analyze
concept maps using KNOT software,
first, students completed concept maps must be saved as a proposition text file
using the Export-Cmap-As feature in the CmapTools
File menu, and then the teacher can easily convert these text files into KNOT proximity files using ALA-Reader software (see Figure 1).
Finally, the proximity files created by ALA-Reader
can be opened and analyzed by KNOT
software.
Figure 1. A mock concept
map (left), the Propositions-as-text file of this map (center) generated using
the Export-Cmap-As feature, and the proximity array (right) generated by ALA-Reader from this text file.
ALA-Reader
software (available at no cost, see: http://www.personal.psu.edu/rbc4/score.htm)
was designed to convert text passages into proximity files, thus allowing
teachers to use KNOT to compare students’
essays to each other and to an expert referent (Clariana & Koul, 2004). The
Mark S feature in ALA-Reader is also
able to convert Cmap
proposition-as-text files directly into KNOT
proximity files, given a list of common terms across all maps.
2.1
Source of the Concepts Maps Used for Analysis
In order to try
this analysis approach, we required multiple similar concept maps on the same topic.
A group of 16 graduate-level students enrolled in one of our curriculum courses
assisted us in this investigation. We decided to parallel an approach used by Lomask,
Baron, Greig, and Harrison (1992) where they converted students’ essays of
science content knowledge into concept maps, and then scored the concept maps
using a quantitative rubric. Working in dyads, our graduate students converted
a student-written 300-word text passage on the heart and circulatory system
into concept maps of this content. We provided them with a list of important
terms and an expert’s concept map of this topic ahead of time. Note that the
members in Dyads 1, 4, 5, 7 and 8 were non-science majors while one of the
members in each of the Dyads 2, 3, and 6 were science majors. Our graduate students
were instructed to make a concept map that accurately represented the text
passage, not their own understanding of the topic. We used this approach in
order to generate concept maps for comparison purposes that are as similar as humanly
possible. However, the eight Dyads produced quite different numbers of
propositions. The non-science majors generally included fewer propositions in
their maps, except for Dyad 1 (Dyad 1 had 42, Dyad 4 had 11, Dyad 5 had 19, Dyad
7 had 15, and Dyad 8 had 15), while the science major dyads had relatively more
propositions in their maps (Dyad 2 had 51, Dyad 3 had 29, and Dyad 6 had 34).
The expert map that they were given ahead of time had 16 propositions (see
Figure 2).
The eight resulting concept maps (one map from each dyad) were
converted to proposition-as-text files and these text files were converted into
eight proximity files using ALA-Reader software (refer
back to Figure 1) using a list of 26 important terms (including synonyms and
metonyms). In addition, the student-written 300-word text passage was also
converted by ALA-Reader into a proximity
file.
Figure 2. An expert
concept map of this topic (note that the ALA-Reader
proposition-as-text format requires
the addition in the top left corner of Concept boxes for the Title, the
expert’s name, and a space).
2.2
KNOT Analysis Results
The proximity
files were converted to network files and then analyzed by KNOT software. The software used the list of 26
important terms, and in this analysis, the values for Minkowski’s r was infinity and q was equal to 25 (e.g., n - 1). In the analysis, each dyad’s concept map network was
compared to every other dyad and to the 300-word text. In addition, KNOT software calculated an ‘average’ proximity file
that is the combination of all 8 dyads, thus providing a measure of collective
group knowledge. The group average was also compared to all of the dyads and to
the 300-word text network. A measure of percent of agreement was calculated by
dividing the number of links in common by the average number of links in the
network pair. The results of this analysis are displayed in Table 1.
|
|
Non-Science Majors |
|
Science Majors |
||||||
|
|
D1 |
D4 |
D5 |
D7 |
D8 |
|
D2 |
D3 |
D6 |
|
Dyad 1 |
-- |
0.08 |
0.13 |
0.07 |
0.07 |
|
0.32 |
0.28 |
0.32 |
|
Dyad 4 |
0.08 |
-- |
0.07 |
0.23 |
0.08 |
|
0.03 |
0.00 |
0.04 |
|
Dyad 5 |
0.13 |
0.07 |
-- |
0.06 |
0.00 |
|
0.51 |
0.50 |
0.45 |
|
Dyad 7 |
0.07 |
0.23 |
0.06 |
-- |
0.00 |
|
0.12 |
0.14 |
0.12 |
|
Dyad 8 |
0.07 |
0.08 |
0.00 |
0.00 |
-- |
|
0.06 |
0.09 |
0.08 |
|
Dyad 2 |
0.32 |
0.03 |
0.51 |
0.12 |
0.06 |
|
-- |
0.70 |
0.68 |
|
Dyad 3 |
0.28 |
0.00 |
0.50 |
0.14 |
0.09 |
|
0.70 |
-- |
0.89 |
|
Dyad 6 |
0.32 |
0.04 |
0.45 |
0.12 |
0.08 |
|
0.68 |
0.89 |
-- |
|
average |
0.27 |
0.19 |
0.34 |
0.17 |
0.28 |
|
0.57 |
0.43 |
0.49 |
|
300-word text |
0.23 |
0.19 |
0.33 |
0.16 |
0.27 |
|
0.53 |
0.39 |
0.45 |
Table 1:
The average percent of agreement
for each pair of concept map networks sorted by major.
We expected
that the percent of agreement between the pairs of dyad concept map networks
would be quite high, since they were all derived from the same 300-word text
passage using the same list of terms. However, this was not the case (see Table
1). The science major dyads were fairly similar (68%, 70%, and 89%), with the
largest percent agreement between Dyads 3 and 6 (89%). For the non-science
dyads, the percent agreement values were considerably lower. The percent agreement
between each dyad and the 300-word text provides another objective measure of
the quality of the dyad maps (in terms of concept links only). Dyad 2 had the
largest percent agreement (e.g., 53%) with the 300-word text network.
Figure 3. The relationship
between the number of propositions in the original dyad concept maps
and the percent agreement with the 300-word text passage network.
Further analysis
indicates that the quality of each dyad’s concept map network is strongly
related to the number of propositions in their concept map (see Figure 3). The
correlation between the number of propositions in the original dyad concept
maps and the percent agreement with the 300-word text passage network is r =
.94, the exception is Dyad 1, that had 42 propositions in the concept map but
only 23% agreement with the 300-word text passage. This data suggests that the
dyads were fairly accurate at translating text propositions into concept map
propositions, but that some dyads were far
more thorough than others. Apparently, prioritizing the importance of
information to include in a concept map is a critical task in concept map
creation.
One other
measure that was calculated is group average. Group average is calculated by KNOT software by averaging together the
cell values of all eight of the proximity array files, and then generating a
network representation of the total group. The resulting group average network
obtained a 62% agreement with the 300-word text network. In addition, the science
major dyads dominated the group average network structure (57%, 43%, and 49%;
see the bottom right side of Table 1), probably because they are more alike and
contain more propositions. Specifically, when KNOT averages propositions across files, common propositions survive
while idiosyncratic propositions drop out.
3 Summary
In their study on the impact of subject matter (content) knowledge on teaching, Hashweh (1987) found that teachers could list many methods and ideas for teaching concepts in their specialty areas but showed misconceptions and misunderstandings about content knowledge in the field that is not their major field. In our study, the quality of the concept map network appears to be related to the participants’ domain knowledge of science: Dyads 2, 3 and 6, with one science major in each dyad, were more accurate in translating text propositions into concept map propositions; Dyads 1, 4, 5, 7, and 8 with no science major, were less accurate in translating text propositions into concept map propositions. While not conclusive, teachers may be less accurate when converting an essay into a concept map if the content of the essay is outside their area of specialty. Our original intent was to describe an easy-to-use approach for quantitatively comparing concept maps. We used our graduate students, a sample of convenience, to convert the 300-word text into concept maps without regard to their background. However, during data analysis, we noted the substantial differences between dyads, and realized the likely importance of dyad prior knowledge in concept map formation, even when their task was only to convert a text to a concept map (specifically directed NOT to use their own knowledge). This finding seems critical in concept map creation.
4 Acknowledgements
This Research Project was provided with graduate student support
time by the Education Department at
References
Clariana, R.B., & Koul, R. (2004). A computer-based approach for translating text into concept map-like representations. In A.J.Canas, J.D.Novak, and F.M.Gonzales, Eds., Concept maps: theory, methodology, technology, vol. 2, in the Proceedings of the First International Conference on Concept Mapping, Pamplona, Spain, Sep 14-17, pp.131-134. Downloaded April 9, 2006 from http://cmc.ihmc.us/papers/cmc2004-045.pdf
Hashweh, M. Z. (1987). Effects of subject matter knowledge in the teaching of biology and physics. Teaching and Teacher Education, 3, 109-120.
Lomask, M., Baron, J., Greig, J., & Harrison, C. (March,
1992). ConnMap:
Novak, J. D. (n.d.). The theory underlying concept maps and how to construct them. Downloaded April 9, 2006 from http://Cmap.coginst.uwf.edu/info/printer.html
Novak, J. D. & Gowin, D. B. (1984) Learning How to Learn.
Schvaneveldt, R. W. (Editor) (1990). Pathfinder associative networks: Studies in knowledge organization.