Recently in Assessment Category

One of the things about Web 2.0 content is that it gets us to begin thinking about content in more creative and less static ways.  Just because something is published to a web page does not mean that it has to stay there...

Let's think about how this could be useful...

We know that some web pages contain content originally published someplace else.  This content reports to this page, sometimes because a tag was used to indicate this relationship.

What if we could put this functionality into the hands of every user?

What if we created one page, a master page that serves as the container looking for certain tags. 

Now, we can have users create a blog entries that include this tag.  These blogs entries would report not only in their blog listing along with all their other blog entries, but more importantly report to the master page as well.

Let's make the master page focus on the learning outcomes for a course or even a program.  Let's use tags that are associated with learning outcomes.  Will this help use organize what we do and share this more efficiently?

In conversations with faculty and staff the term 'curriculum map' has been coming up lately.  I thought it would be handy to have a reference that I could point that helps to clarify what I am thinking about when it comes to mapping a curriculum.

Curriculum maps lay out courses and/or experiences on one hand and the program learning outcomes or accreditation standards on the other (below).  The X's represent key assessments which will represent student achievement of an outcome.  So, for instance, in 401 there are two key assessment opportunities where students can demonstrate achievement of outcomes A and C.

This key assessment may be a paper, exam, project or other example of student work. This agreed assignment between the key assessment and the course or experiences in which this may take place is essential.  The map lays out how each course or experience contributes to the overall program of study. This is also the information that is used by an assessment system to not only store but track where this evidence of learning is coming from and which outcome or standard it supports.

Over the years Penn State's involvement in the coalition has provided us with opportunities for moving thinking into action.  Do we use our discussion and notes about integrative learning? Here is an example of where we are now:

"Integrative learners do not separate learning from life..."

The integrated learning experience presupposes purpose, involves reflection and personal connection, while at the same time implies an understanding of what the important attributes, values, understandings that are to be learned.  From the beginning of our involvement NCEPR our goal at Penn State has been has been to find out whether a structured e-portfolio system, designed to communicate high expectations and intended learning outcomes could have an impact on student engagement and learning.

In other words, how do we share a 'big picture' of learning and then place our students within this picture such that this position serves as a scaffold for finding new understandings, promoting meaning making that leads to self-authorship, concepts that Kegan and Baxter-Magolda advocate.  We know we need to invite prior experience to play an important part in shaping this experience but also realize that students bring messy pictures into this arena.  The challenge lies in the gap between where students are and where our big picture implies they should be.

What is the developmental pattern for how students can be brought through this integrative process? A simple advanced organizer is not enough.  How do we infuse structure while at the same time allow for a flexibility that cultivates students' becoming 'architects of their own education'?

We've moved through a variety of unsatisfactory solutions and frustration surrounding technology as 'a moving target' is getting old, but we are learning.  Nonetheless, we are excited about our most recent deployment, MovableType4, to support Blogging as e-Portfolios.  We don't think of this as a new service because of how nicely it integrates into the technology infrastructure already in place.  Further, it has dramatically reduced any learning curve associated with online publishing, students are comfortable with its capabilities and convenience. But more importantly it allows us to 1) include templates that state program learning outcomes, 2) use tags to associate artifacts with these outcomes, 3) use tags to provide intellectual access to content previously not possible, and 4) aggregate and 'pack up' specific program outcome related evidence for assessment or evaluation purposes.

We are hopeful that this most recent exploration into the use of the MovableType blogging platform will help us in delivering the three curricula we have been talking about: the delivered (teachers' goals for students), the experienced (students' perceptions of the learning experience) and the lived (personalization of how this is experienced in life). The unobtrusiveness of this solution is perhaps it greatest feature because looking back at earlier solutions we've thought that perhaps the more we get out of the way, the better we'll serve our students.

As Kathleen Yancey noted at the Waterloo meetings, "Biggest take-way: be very intentional with communication." 

A good chunk of the initial energy of Penn State's e-portfolio initiative focused on supporting e-portfolio activity by writing specifications for and finding, modifying or developing an enterprise solution that would both provide a student-centered e-portfolio interface to support learning but also provide faculty and administrators with tools that could tap this evidence of learning to support program evaluation.  Based primarily on the contrary nature of both of these purposes, this 'silver bullet' has yet to be found. 

Consequently, Penn State seems to be moving in two different directions.  Deployment of the Penn State Blog (Moveable Type 4) provides users with a simple yet flexible interface for supporting the learning, connecting and sharing of collegiate experience. 

But what about assessment?  Has it been eliminated from the e-portfolio picture?

While there is a great deal that faculty can learn about the developmental work that takes place as a student's e-portfolio or blog takes shape and develops over time, capturing all of this data from all students for the purposes of determining program effectiveness would be unnecessarily messy.

Teaming with faculty from the Department of Counselor Education, Counseling Psychology and Rehabilitation Services the e-Portfolio Initiative sought local Penn State resources to help address their program evaluation and accreditation needs.  Having established a curriculum map  that articulates key assessment activities related to accreditation standards, WebLion, Penn State's customization of a Plone based open source content management software was asked to help develop a solution.  Very quickly, developers were able to create an online repository that faculty can use to upload, store and tag evidence of student learning based on these standards.  What was previously an inefficient and often redundant paper-based process, (this program, for example, must address the needs of three accreditation agencies - NCATE, CACREP and PDE), has become a more streamlined assessment management system that can be queried at any time to review progress on key learning outcomes by faculty, assessment coordinators and when ready outside auditors.



An important distinction needs to be made with regards to this system. Rather than automatically grabbing all student submissions from an ANGEL dropbox for instance, faculty select representative examples of evidence from the key assessment activities that take place in the courses that they teach and only upload these examples to the system.  For instance, it may seem reasonable to select nine examples of a key assessment activity as a representative sample of student work (three superior, three average and three poor examples) from each course section.  Over the course of an accreditation cycle, it is assumed that enough evidence of student learning has accumulated such that justifications for what is taught and how can be made.

For more information about WebLion and Assessment Management contact Mike Halm.